[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:40 dist-test-slave-gkw7 krb5kdc[32675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:40 dist-test-slave-gkw7 krb5kdc[32675](info): set up 2 sockets
Aug 07 15:14:40 dist-test-slave-gkw7 krb5kdc[32675](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 15:14:41.996809 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:14:42.133996 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45095
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:45095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:42.248518 32693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:42.248867 32693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:42.248958 32693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:42.253002 32693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:42.253098 32693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:42.253139 32693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:42.253176 32693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:42.253209 32693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:42.258529 32693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:45095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:42.259719 32693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:42.260864 32693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:42.270153 32706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:42.270148 32705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:42.270195 32710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:42.270211 32693 server_base.cc:1034] running on GCE node
I20260807 15:14:42.271090 32693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:42.272308 32693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:42.273514 32693 hybrid_clock.cc:648] HybridClock initialized: now 1786115682273488 us; error 60 us; skew 500 ppm
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:42.276712 32693 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:14:42.277777 32693 webserver.cc:460] Webserver started at http://127.31.230.254:42451/ using document root <none> and password file <none>
I20260807 15:14:42.278393 32693 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:42.278496 32693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:42.278767 32693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:42.281219 32693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "acb74dc091194abaa63d8c9ae3587673"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "360a7685ada53ff551836584be7806ef"
server_key_iv: "5dae253ab25103858ac5e057109b1895"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.281944 32693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "acb74dc091194abaa63d8c9ae3587673"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "360a7685ada53ff551836584be7806ef"
server_key_iv: "5dae253ab25103858ac5e057109b1895"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.286062 32693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:42.288826 32718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.290333 32693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:42.290483 32693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "acb74dc091194abaa63d8c9ae3587673"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "360a7685ada53ff551836584be7806ef"
server_key_iv: "5dae253ab25103858ac5e057109b1895"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.290604 32693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:42.300540 32693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:42.304031 32693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:42.304275 32693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:42.312350 32693 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:45095
I20260807 15:14:42.312418 325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:45095 every 8 connection(s)
I20260807 15:14:42.313454 32693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:42.316457 327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:42.322057 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 32693
I20260807 15:14:42.322189 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:14:42.322639 327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673: Bootstrap starting.
I20260807 15:14:42.323598 32667 external_mini_cluster.cc:1428] Setting key 1c205caf878f15df7ba94fae94522cc5
I20260807 15:14:42.325214 327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:42.326135 327 log.cc:824] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:42.328230 327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673: No bootstrap required, opened a new log
I20260807 15:14:42.332396 327 raft_consensus.cc:348] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb74dc091194abaa63d8c9ae3587673" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45095 } }
I20260807 15:14:42.332695 327 raft_consensus.cc:374] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:42.332773 327 raft_consensus.cc:729] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb74dc091194abaa63d8c9ae3587673, State: Initialized, Role: FOLLOWER
I20260807 15:14:42.333232 327 consensus_queue.cc:260] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb74dc091194abaa63d8c9ae3587673" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45095 } }
I20260807 15:14:42.333360 327 raft_consensus.cc:388] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:42.333451 327 raft_consensus.cc:482] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:42.333609 327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:42.334553 327 raft_consensus.cc:504] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb74dc091194abaa63d8c9ae3587673" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45095 } }
I20260807 15:14:42.334887 327 leader_election.cc:302] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acb74dc091194abaa63d8c9ae3587673; no voters:
I20260807 15:14:42.335203 327 leader_election.cc:288] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:42.335378 335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:42.335602 335 raft_consensus.cc:686] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [term 1 LEADER]: Becoming Leader. State: Replica: acb74dc091194abaa63d8c9ae3587673, State: Running, Role: LEADER
I20260807 15:14:42.335901 335 consensus_queue.cc:237] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb74dc091194abaa63d8c9ae3587673" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45095 } }
I20260807 15:14:42.336496 327 sys_catalog.cc:564] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:42.338903 336 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acb74dc091194abaa63d8c9ae3587673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb74dc091194abaa63d8c9ae3587673" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45095 } } }
I20260807 15:14:42.339025 336 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:42.339254 337 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acb74dc091194abaa63d8c9ae3587673. Latest consensus state: current_term: 1 leader_uuid: "acb74dc091194abaa63d8c9ae3587673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb74dc091194abaa63d8c9ae3587673" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45095 } } }
I20260807 15:14:42.339416 337 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673 [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:42.341930 354 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:42.342042 354 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:42.342123 355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:42.342931 355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:42.349025 355 catalog_manager.cc:1347] Generated new cluster ID: 4a540523a459415d9f5f23e259b7f2d6
I20260807 15:14:42.349105 355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:42.351006 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:42.326796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.31.230.254:45095)
0807 15:14:42.327272 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:42.327294 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:42.328541 (+ 1247us) server_negotiation.cc:407] Connection header received
0807 15:14:42.331932 (+ 3391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:42.331955 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:42.332343 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:42.332708 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:42.335305 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.336805 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.338329 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.339141 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.339320 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:42.344446 (+ 5126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:42.344454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:42.344470 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:42.344503 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:42.346937 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:42.347648 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:42.347653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:42.347658 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:42.347777 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:42.348170 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:42.348173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:42.348174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:42.348606 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:42.348852 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:42.349749 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 15:14:42.350061 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":109,"threads_started":1}
I20260807 15:14:42.357016 355 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:42.358011 355 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:42.364215 355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acb74dc091194abaa63d8c9ae3587673: Generated new TSK 0
I20260807 15:14:42.364938 355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:14:42.405439 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45095
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:42.538136 365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:42.538435 365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:42.538527 365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:42.542780 365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:42.542850 365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:42.542943 365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:14:42.548251 365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45095
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:42.549463 365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:42.550566 365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:42.561035 372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:42.561102 365 server_base.cc:1034] running on GCE node
W20260807 15:14:42.561049 373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:42.561060 377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:42.561753 365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:42.562471 365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:42.563645 365 hybrid_clock.cc:648] HybridClock initialized: now 1786115682563616 us; error 38 us; skew 500 ppm
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:42.566779 365 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:14:42.567819 365 webserver.cc:460] Webserver started at http://127.31.230.193:39267/ using document root <none> and password file <none>
I20260807 15:14:42.568490 365 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:42.568560 365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:42.568866 365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:42.571238 365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "03a9c55c40ee479cb7e12bfad39d1442"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "1d7f71cf0458112705cfa6daf8404d23"
server_key_iv: "a6655f09a61fff2fd27b6f42a422425a"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.571879 365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "03a9c55c40ee479cb7e12bfad39d1442"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "1d7f71cf0458112705cfa6daf8404d23"
server_key_iv: "a6655f09a61fff2fd27b6f42a422425a"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.576174 365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:14:42.578873 386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.580214 365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:42.580348 365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "03a9c55c40ee479cb7e12bfad39d1442"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "1d7f71cf0458112705cfa6daf8404d23"
server_key_iv: "a6655f09a61fff2fd27b6f42a422425a"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.580468 365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:42.597247 365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:42.600677 365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:42.600924 365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:42.601678 365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:42.602654 365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:42.602720 365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.602807 365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:42.602844 365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.613610 556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:34781 every 8 connection(s)
I20260807 15:14:42.613642 365 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:34781
I20260807 15:14:42.614732 365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:42.624303 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 365
I20260807 15:14:42.624449 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:42.624745 32667 external_mini_cluster.cc:1428] Setting key 37555be52e723b0d2fe58cf0d26a6709
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:42.631419 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:42.617141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:43159 (local address 127.31.230.254:45095)
0807 15:14:42.617345 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:42.617351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:42.618482 (+ 1131us) server_negotiation.cc:407] Connection header received
0807 15:14:42.619955 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:42.619958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:42.620014 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:42.620119 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:42.621519 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.622170 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.623374 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.623623 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.623710 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:42.626693 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:42.626698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:42.626700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:42.626729 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:42.628613 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:42.629370 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:42.629374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:42.629375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:42.629483 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:42.630105 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:42.630108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:42.630109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:42.630328 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:42.630427 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:42.631155 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 15:14:42.631286 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 15:14:42.632748 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:42.617652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45095 (local address 127.31.230.193:43159)
0807 15:14:42.618269 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:42.618333 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:42.619717 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:42.620406 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:42.620418 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:42.620829 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:42.621352 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:42.621366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.622299 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.622302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:42.623252 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:42.623261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.623773 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.623776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:42.623925 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:42.624718 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:42.624736 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:42.626509 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:42.628774 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:42.628782 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:42.628802 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:42.629238 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:42.629720 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:42.629726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:42.629731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:42.629950 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:42.630462 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:42.630472 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:42.630850 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 15:14:42.631088 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 15:14:42.631503 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":144,"threads_started":1}
I20260807 15:14:42.634339 557 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45095
I20260807 15:14:42.634639 557 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:42.635317 557 heartbeater.cc:511] Master 127.31.230.254:45095 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:14:42.637311 32745 ts_manager.cc:194] Registered new tserver with Master: 03a9c55c40ee479cb7e12bfad39d1442 (127.31.230.193:34781)
I20260807 15:14:42.638769 32745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:43159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:14:42.658027 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45095
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:42.776470 564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:42.776808 564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:42.776894 564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:42.780920 564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:42.781032 564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:42.781131 564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:14:42.786713 564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45095
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:42.787845 564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:42.788962 564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:42.799263 570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:42.799268 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:42.799379 564 server_base.cc:1034] running on GCE node
W20260807 15:14:42.800015 571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:42.800513 564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:42.801328 564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:42.802544 564 hybrid_clock.cc:648] HybridClock initialized: now 1786115682802530 us; error 49 us; skew 500 ppm
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:42.805497 564 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:14:42.806694 564 webserver.cc:460] Webserver started at http://127.31.230.194:37217/ using document root <none> and password file <none>
I20260807 15:14:42.807309 564 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:42.807408 564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:42.807643 564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:42.809832 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "78232d3b6a1d478ebadef89dcb1bebd6"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "a549aa372ad74a6d5f00c0798dda38ab"
server_key_iv: "d833626b2e324426464460e087ef05b6"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.810436 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "78232d3b6a1d478ebadef89dcb1bebd6"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "a549aa372ad74a6d5f00c0798dda38ab"
server_key_iv: "d833626b2e324426464460e087ef05b6"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.814301 564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:42.816890 583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.818276 564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:42.818418 564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "78232d3b6a1d478ebadef89dcb1bebd6"
format_stamp: "Formatted at 2026-08-07 15:14:42 on dist-test-slave-gkw7"
server_key: "a549aa372ad74a6d5f00c0798dda38ab"
server_key_iv: "d833626b2e324426464460e087ef05b6"
server_key_version: "encryptionkey@0"
I20260807 15:14:42.818537 564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:42.858875 564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:42.862104 564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:42.862339 564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:42.862978 564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:42.863893 564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:42.863983 564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.864059 564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:42.864110 564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:42.873764 564 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:36235
I20260807 15:14:42.873792 748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:36235 every 8 connection(s)
I20260807 15:14:42.874714 564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:42.876286 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 564
I20260807 15:14:42.876399 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:42.876762 32667 external_mini_cluster.cc:1428] Setting key 8f63801d00fd6047752aea53a7f01281
Aug 07 15:14:42 dist-test-slave-gkw7 krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115682, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:14:42.890213 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:42.876948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59997 (local address 127.31.230.254:45095)
0807 15:14:42.877086 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:42.877090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:42.878198 (+ 1108us) server_negotiation.cc:407] Connection header received
0807 15:14:42.879787 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:42.879790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:42.879844 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:42.879928 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:42.881698 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.882238 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.883423 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.883650 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.883813 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:42.886528 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:42.886532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:42.886535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:42.886568 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:42.888042 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:42.888663 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:42.888665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:42.888667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:42.888752 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:42.889066 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:42.889068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:42.889069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:42.889321 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:42.889423 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:42.889974 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 15:14:42.890083 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:14:42.891310 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:42.877450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45095 (local address 127.31.230.194:59997)
0807 15:14:42.878023 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:42.878083 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:42.879571 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:42.880114 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:42.880121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:42.880735 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:42.881462 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:42.881486 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.882358 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.882360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:42.883279 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:42.883288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:42.883856 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:42.883860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:42.884029 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:42.884776 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:42.884792 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:42.886358 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:42.888192 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:42.888198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:42.888216 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:42.888563 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:42.888838 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:42.888840 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:42.888842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:42.888977 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:42.889425 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:42.889430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:42.889744 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 15:14:42.889904 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 15:14:42.890201 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":131,"threads_started":1}
I20260807 15:14:42.892561 749 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45095
I20260807 15:14:42.892841 749 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:42.893416 749 heartbeater.cc:511] Master 127.31.230.254:45095 requested a full tablet report, sending...
I20260807 15:14:42.894608 32745 ts_manager.cc:194] Registered new tserver with Master: 78232d3b6a1d478ebadef89dcb1bebd6 (127.31.230.194:36235)
I20260807 15:14:42.895308 32745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:59997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:14:42.909093 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45095
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:43.027447 757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:43.027765 757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:43.027846 757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:43.032034 757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:43.032137 757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:43.032234 757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:14:43.037889 757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45095
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:43.039041 757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:43.040122 757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:43.050379 766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:43.050489 757 server_base.cc:1034] running on GCE node
W20260807 15:14:43.050424 764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:43.050390 763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:43.051082 757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:43.051749 757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:43.052918 757 hybrid_clock.cc:648] HybridClock initialized: now 1786115683052893 us; error 38 us; skew 500 ppm
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115683, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:43.055864 757 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:14:43.056792 757 webserver.cc:460] Webserver started at http://127.31.230.195:33251/ using document root <none> and password file <none>
I20260807 15:14:43.057404 757 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:43.057503 757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:43.057786 757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:43.059955 757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "54938d1426864bec8d69d7fde9094da3"
format_stamp: "Formatted at 2026-08-07 15:14:43 on dist-test-slave-gkw7"
server_key: "5f7506904b393edf3700ea33ff28362b"
server_key_iv: "f82fd753b2bc192be3d9a0d6b456a7cf"
server_key_version: "encryptionkey@0"
I20260807 15:14:43.060635 757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "54938d1426864bec8d69d7fde9094da3"
format_stamp: "Formatted at 2026-08-07 15:14:43 on dist-test-slave-gkw7"
server_key: "5f7506904b393edf3700ea33ff28362b"
server_key_iv: "f82fd753b2bc192be3d9a0d6b456a7cf"
server_key_version: "encryptionkey@0"
I20260807 15:14:43.064621 757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:14:43.067251 776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:43.068476 757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:43.068620 757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "54938d1426864bec8d69d7fde9094da3"
format_stamp: "Formatted at 2026-08-07 15:14:43 on dist-test-slave-gkw7"
server_key: "5f7506904b393edf3700ea33ff28362b"
server_key_iv: "f82fd753b2bc192be3d9a0d6b456a7cf"
server_key_version: "encryptionkey@0"
I20260807 15:14:43.068738 757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:43.080364 757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:43.083314 757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:43.083523 757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:43.084136 757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:43.085086 757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:43.085160 757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:43.085238 757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:43.085287 757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:43.094576 757 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:44485
I20260807 15:14:43.094609 936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:44485 every 8 connection(s)
I20260807 15:14:43.095504 757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:43.096419 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 757
I20260807 15:14:43.096524 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:43.096748 32667 external_mini_cluster.cc:1428] Setting key 755f2cba611314f51d2ac019d5021c01
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115683, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:43.110972 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:43.098582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:37925 (local address 127.31.230.254:45095)
0807 15:14:43.098731 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:43.098736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:43.099815 (+ 1079us) server_negotiation.cc:407] Connection header received
0807 15:14:43.101161 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:43.101164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:43.101216 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:43.101292 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:43.102534 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.103166 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.104366 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.104595 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.104706 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:43.107175 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:43.107179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:43.107181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:43.107205 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:43.108569 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:43.109150 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:43.109154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:43.109156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:43.109243 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:43.109671 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:43.109674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:43.109676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:43.109867 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:43.109980 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:43.110657 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 15:14:43.110789 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:14:43.111920 939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:43.099175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45095 (local address 127.31.230.195:37925)
0807 15:14:43.099679 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:43.099722 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:43.100960 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:43.101456 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:43.101463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:43.101902 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:43.102386 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:43.102401 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.103303 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.103306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:43.104233 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:43.104243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.104747 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.104751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:43.104925 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:43.105477 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:43.105492 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:43.106993 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:43.108721 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:43.108727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:43.108743 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:43.109026 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:43.109342 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:43.109345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:43.109348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:43.109495 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:43.109981 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:43.109989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:43.110298 (+ 309us) client_negotiation.cc:769] Sending connection context
0807 15:14:43.110533 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 15:14:43.110907 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":117,"threads_started":1}
I20260807 15:14:43.112974 937 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45095
I20260807 15:14:43.113248 937 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:43.113806 937 heartbeater.cc:511] Master 127.31.230.254:45095 requested a full tablet report, sending...
I20260807 15:14:43.114856 32745 ts_manager.cc:194] Registered new tserver with Master: 54938d1426864bec8d69d7fde9094da3 (127.31.230.195:44485)
I20260807 15:14:43.115393 32745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:37925
I20260807 15:14:43.123347 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:14:43.151204 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:43.141697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.31.230.193:34781)
0807 15:14:43.142034 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:43.142042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:43.142077 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:14:43.142200 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:43.142206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:43.142412 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:43.142574 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:43.143262 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.144029 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.144876 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.145255 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.145369 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:43.147764 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:43.147773 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:43.147788 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:43.147812 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:43.149323 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:43.149842 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:43.149853 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:43.149855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:43.149937 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:43.150229 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:43.150233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:43.150234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:43.150477 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:43.150708 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:43.150885 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 15:14:43.151047 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
W20260807 15:14:43.152253 430 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42678
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:14:43.180200 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:43.171246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.31.230.193:34781)
0807 15:14:43.171420 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:43.171425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:43.171569 (+ 144us) server_negotiation.cc:407] Connection header received
0807 15:14:43.171756 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:43.171759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:43.171802 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:43.171888 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:43.172680 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.173200 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.174125 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:43.174372 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:43.174682 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:43.177144 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:43.177148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:43.177150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:43.177174 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:43.178482 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:43.178915 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:43.178918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:43.178920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:43.179006 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:43.179302 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:43.179307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:43.179309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:43.179495 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:43.179653 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:43.179892 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 15:14:43.180020 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:14:43.181265 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 365
I20260807 15:14:43.192886 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 564
I20260807 15:14:43.224535 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 757
I20260807 15:14:43.243927 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 32693
2026-08-07T15:14:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2496 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[955](info): set up 2 sockets
Aug 07 15:14:43 dist-test-slave-gkw7 krb5kdc[955](info): commencing operation
krb5kdc: starting...
W20260807 15:14:44.484357 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:14:44 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:14:44.621830 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33695
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:33695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:44.736735 973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:44.737017 973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:44.737058 973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:44.740989 973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:44.741047 973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:44.741067 973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:44.741084 973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:44.741099 973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:44.746371 973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:33695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:44.747423 973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:44.748432 973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:44.757114 982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:44.757175 987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:44.757175 980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:44.757339 973 server_base.cc:1034] running on GCE node
I20260807 15:14:44.758136 973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:44.759301 973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:44.760489 973 hybrid_clock.cc:648] HybridClock initialized: now 1786115684760457 us; error 48 us; skew 500 ppm
Aug 07 15:14:44 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115684, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:44.763341 973 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:14:44.764271 973 webserver.cc:460] Webserver started at http://127.31.230.254:35809/ using document root <none> and password file <none>
I20260807 15:14:44.764846 973 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:44.764946 973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:44.765192 973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:44.767292 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "4c5fca2503304249ad756298cb053a3d"
format_stamp: "Formatted at 2026-08-07 15:14:44 on dist-test-slave-gkw7"
server_key: "6e07c127114d02cdb25ab6661004952f"
server_key_iv: "26457c97c10b0cda69d0b9c7a8117c3d"
server_key_version: "encryptionkey@0"
I20260807 15:14:44.767870 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "4c5fca2503304249ad756298cb053a3d"
format_stamp: "Formatted at 2026-08-07 15:14:44 on dist-test-slave-gkw7"
server_key: "6e07c127114d02cdb25ab6661004952f"
server_key_iv: "26457c97c10b0cda69d0b9c7a8117c3d"
server_key_version: "encryptionkey@0"
I20260807 15:14:44.771822 973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:14:44.774317 997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:44.775795 973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:44.775959 973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "4c5fca2503304249ad756298cb053a3d"
format_stamp: "Formatted at 2026-08-07 15:14:44 on dist-test-slave-gkw7"
server_key: "6e07c127114d02cdb25ab6661004952f"
server_key_iv: "26457c97c10b0cda69d0b9c7a8117c3d"
server_key_version: "encryptionkey@0"
I20260807 15:14:44.776120 973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:44.787427 973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:44.790730 973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:44.790941 973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:44.798732 973 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:33695
I20260807 15:14:44.798755 1074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:33695 every 8 connection(s)
I20260807 15:14:44.799736 973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:44.802580 1075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:44.809088 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d: Bootstrap starting.
I20260807 15:14:44.809837 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 973
I20260807 15:14:44.809959 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:14:44.810238 32667 external_mini_cluster.cc:1428] Setting key 442deb0d3b6728e798709c4c3a2ebf05
I20260807 15:14:44.812155 1075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:44.812917 1075 log.cc:824] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:44.815207 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d: No bootstrap required, opened a new log
Aug 07 15:14:44 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:44.819941 1075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5fca2503304249ad756298cb053a3d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33695 } }
I20260807 15:14:44.820197 1075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:44.820256 1075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5fca2503304249ad756298cb053a3d, State: Initialized, Role: FOLLOWER
I20260807 15:14:44.820761 1075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5fca2503304249ad756298cb053a3d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33695 } }
I20260807 15:14:44.820919 1075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:44.820990 1075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:44.821144 1075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:44.822202 1075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5fca2503304249ad756298cb053a3d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33695 } }
I20260807 15:14:44.822564 1075 leader_election.cc:302] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c5fca2503304249ad756298cb053a3d; no voters:
I20260807 15:14:44.822965 1075 leader_election.cc:288] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:44.823043 1080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:44.823272 1080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [term 1 LEADER]: Becoming Leader. State: Replica: 4c5fca2503304249ad756298cb053a3d, State: Running, Role: LEADER
I20260807 15:14:44.823578 1080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5fca2503304249ad756298cb053a3d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33695 } }
I20260807 15:14:44.824401 1075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:44.825488 1083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c5fca2503304249ad756298cb053a3d. Latest consensus state: current_term: 1 leader_uuid: "4c5fca2503304249ad756298cb053a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5fca2503304249ad756298cb053a3d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33695 } } }
I20260807 15:14:44.825491 1085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c5fca2503304249ad756298cb053a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5fca2503304249ad756298cb053a3d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33695 } } }
I20260807 15:14:44.825657 1083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:44.825664 1085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:44.826555 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:44.811600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.31.230.254:33695)
0807 15:14:44.812048 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:44.812063 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:44.812097 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:14:44.812819 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:44.812837 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:44.813223 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:44.813639 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:44.814568 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:44.815615 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:44.816776 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:44.817231 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:44.817354 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:44.820315 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:44.820321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:44.820340 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:44.820373 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:44.822405 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:44.822928 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:44.822934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:44.822940 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:44.823151 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:44.823516 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:44.823520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:44.823522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:44.824019 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:44.824283 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:44.824897 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 15:14:44.825228 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":129,"threads_started":1}
I20260807 15:14:44.827037 1094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:44.830227 1094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:44.835577 1094 catalog_manager.cc:1347] Generated new cluster ID: bcd73a3e23024d1fba0f4801219a32c6
I20260807 15:14:44.835659 1094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:44.845980 1094 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:44.847213 1094 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:44.855149 1094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c5fca2503304249ad756298cb053a3d: Generated new TSK 0
I20260807 15:14:44.855913 1094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:14:44.888707 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33695
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:45.007179 1110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:45.007454 1110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:45.007500 1110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:45.011658 1110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:45.011739 1110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:45.011838 1110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:14:45.017362 1110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33695
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:45.018558 1110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:45.019658 1110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:45.029915 1110 server_base.cc:1034] running on GCE node
W20260807 15:14:45.029798 1121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:45.029810 1123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:45.029829 1125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:45.030488 1110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:45.031159 1110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:45.032348 1110 hybrid_clock.cc:648] HybridClock initialized: now 1786115685032315 us; error 45 us; skew 500 ppm
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:45.035336 1110 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:14:45.036253 1110 webserver.cc:460] Webserver started at http://127.31.230.193:36789/ using document root <none> and password file <none>
I20260807 15:14:45.036793 1110 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:45.036854 1110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:45.037037 1110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:45.039103 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "2ff6a87af1734061b15f1bbfd8538d8b"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "c89da43e65ad23c4a9b701bd4107c9ff"
server_key_iv: "3863362bb64e956e67e2f9b699af94a8"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.039610 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "2ff6a87af1734061b15f1bbfd8538d8b"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "c89da43e65ad23c4a9b701bd4107c9ff"
server_key_iv: "3863362bb64e956e67e2f9b699af94a8"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.043510 1110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 15:14:45.046037 1140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.047322 1110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:45.047437 1110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "2ff6a87af1734061b15f1bbfd8538d8b"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "c89da43e65ad23c4a9b701bd4107c9ff"
server_key_iv: "3863362bb64e956e67e2f9b699af94a8"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.047525 1110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:45.071193 1110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:45.074716 1110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:45.074904 1110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:45.075466 1110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:45.076347 1110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:45.076399 1110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.076442 1110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:45.076460 1110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.087286 1110 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:41057
I20260807 15:14:45.087349 1312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:41057 every 8 connection(s)
I20260807 15:14:45.088404 1110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:45.096592 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 1110
I20260807 15:14:45.096714 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:45.097007 32667 external_mini_cluster.cc:1428] Setting key e2b78e144f8709ee839d2b976b2de3d5
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:45.103772 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.090544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:35819 (local address 127.31.230.254:33695)
0807 15:14:45.090696 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.090700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.091628 (+ 928us) server_negotiation.cc:407] Connection header received
0807 15:14:45.092996 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.092999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.093058 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.093164 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.094431 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.094981 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.096129 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.096447 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.096570 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.099721 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.099724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.099727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.099751 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.101613 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.102125 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.102131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.102134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.102244 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.102636 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.102639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.102640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.102821 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.102906 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.103542 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.103647 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:14:45.105021 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.090984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33695 (local address 127.31.230.193:35819)
0807 15:14:45.091481 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.091525 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.092773 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.093347 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.093353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.093794 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:45.094283 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.094297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.095113 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.095121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.095999 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.096009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.096643 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.096647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.096893 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.097597 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:45.097642 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:45.099548 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:45.101729 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:45.101735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:45.101748 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:45.102011 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:45.102343 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:45.102345 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:45.102346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:45.102521 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:45.102934 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:45.102943 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:45.103282 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.103497 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.103863 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
I20260807 15:14:45.106433 1313 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33695
I20260807 15:14:45.106693 1313 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:45.107199 1313 heartbeater.cc:511] Master 127.31.230.254:33695 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:14:45.109001 1023 ts_manager.cc:194] Registered new tserver with Master: 2ff6a87af1734061b15f1bbfd8538d8b (127.31.230.193:41057)
I20260807 15:14:45.110417 1023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:35819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:14:45.129441 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33695
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:45.246013 1319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:45.246304 1319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:45.246383 1319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:45.250432 1319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:45.250486 1319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:45.250592 1319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:14:45.256129 1319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33695
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:45.257184 1319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:45.258292 1319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:45.268143 1328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:45.268132 1330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:45.268332 1334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:45.268498 1319 server_base.cc:1034] running on GCE node
I20260807 15:14:45.268997 1319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:45.269917 1319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:45.271173 1319 hybrid_clock.cc:648] HybridClock initialized: now 1786115685271128 us; error 58 us; skew 500 ppm
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:45.275384 1319 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:14:45.276805 1319 webserver.cc:460] Webserver started at http://127.31.230.194:43789/ using document root <none> and password file <none>
I20260807 15:14:45.277606 1319 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:45.277717 1319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:45.278064 1319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:45.281075 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "d84d0c08515642a88473cff565419804"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "9041fbaaf783e7af9a86e29a0acd39bf"
server_key_iv: "197945fb91cf2f89b51e468dff1f7ca7"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.281847 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "d84d0c08515642a88473cff565419804"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "9041fbaaf783e7af9a86e29a0acd39bf"
server_key_iv: "197945fb91cf2f89b51e468dff1f7ca7"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.286219 1319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:45.288990 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.290356 1319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:45.290459 1319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "d84d0c08515642a88473cff565419804"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "9041fbaaf783e7af9a86e29a0acd39bf"
server_key_iv: "197945fb91cf2f89b51e468dff1f7ca7"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.290540 1319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:45.310451 1319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:45.313335 1319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:45.313493 1319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:45.314137 1319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:45.314935 1319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:45.314982 1319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.315021 1319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:45.315037 1319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.323848 1319 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:41165
I20260807 15:14:45.323877 1506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:41165 every 8 connection(s)
I20260807 15:14:45.324908 1319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:45.327082 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 1319
I20260807 15:14:45.327193 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:45.327454 32667 external_mini_cluster.cc:1428] Setting key ba6bd180dda9cd85b0acc8b020e71395
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:14:45.339704 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.327004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:40825 (local address 127.31.230.254:33695)
0807 15:14:45.327195 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.327201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.328172 (+ 971us) server_negotiation.cc:407] Connection header received
0807 15:14:45.329605 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.329611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.329675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.329772 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.330919 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.331450 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.332606 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.332843 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.332964 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.335792 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.335797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.335802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.335838 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.337564 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.338076 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.338080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.338082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.338180 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.338574 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.338578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.338579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.338790 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.338873 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.339381 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.339523 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:14:45.340545 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.327511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33695 (local address 127.31.230.194:40825)
0807 15:14:45.328018 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.328071 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.329370 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.329928 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.329935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.330299 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:45.330772 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.330786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.331579 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.331582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.332479 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.332488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.333003 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.333007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.333189 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.333876 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:45.333891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:45.335613 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:45.337693 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:45.337697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:45.337710 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:45.337972 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:45.338312 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:45.338318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:45.338320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:45.338468 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:45.338885 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:45.338890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:45.339116 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.339318 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.339661 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:14:45.341950 1507 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33695
I20260807 15:14:45.342209 1507 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:45.342721 1507 heartbeater.cc:511] Master 127.31.230.254:33695 requested a full tablet report, sending...
I20260807 15:14:45.343829 1023 ts_manager.cc:194] Registered new tserver with Master: d84d0c08515642a88473cff565419804 (127.31.230.194:41165)
I20260807 15:14:45.344421 1023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:40825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:14:45.357610 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33695
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:45.475391 1515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:45.475661 1515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:45.475706 1515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:45.479830 1515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:45.479895 1515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:45.479956 1515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:14:45.485468 1515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33695
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:45.486544 1515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:45.487545 1515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:45.497790 1525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:45.497848 1515 server_base.cc:1034] running on GCE node
W20260807 15:14:45.497776 1523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:45.497792 1522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:45.498450 1515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:45.499154 1515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:45.500382 1515 hybrid_clock.cc:648] HybridClock initialized: now 1786115685500342 us; error 48 us; skew 500 ppm
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:45.503480 1515 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:14:45.504426 1515 webserver.cc:460] Webserver started at http://127.31.230.195:34811/ using document root <none> and password file <none>
I20260807 15:14:45.505005 1515 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:45.505101 1515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:45.505350 1515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:45.507513 1515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "55aa6e6d88974a7e9077bf2c84f91413"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "b0480538110c8c179e75a53ef821a4f5"
server_key_iv: "d400dcf1704959b7417929f652829334"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.508090 1515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "55aa6e6d88974a7e9077bf2c84f91413"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "b0480538110c8c179e75a53ef821a4f5"
server_key_iv: "d400dcf1704959b7417929f652829334"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.512552 1515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 15:14:45.515107 1534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.516367 1515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:45.516510 1515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "55aa6e6d88974a7e9077bf2c84f91413"
format_stamp: "Formatted at 2026-08-07 15:14:45 on dist-test-slave-gkw7"
server_key: "b0480538110c8c179e75a53ef821a4f5"
server_key_iv: "d400dcf1704959b7417929f652829334"
server_key_version: "encryptionkey@0"
I20260807 15:14:45.516621 1515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:45.539090 1515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:45.542188 1515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:45.542407 1515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:45.543045 1515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:45.544047 1515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:45.544130 1515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.544229 1515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:45.544279 1515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:45.554133 1515 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:43985
I20260807 15:14:45.554153 1691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:43985 every 8 connection(s)
I20260807 15:14:45.555080 1515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:45.565207 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 1515
I20260807 15:14:45.565415 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:45.565755 32667 external_mini_cluster.cc:1428] Setting key 9a622f123b26a63db45f8f14d20b8edf
I20260807 15:14:45.569357 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.556973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:55149 (local address 127.31.230.254:33695)
0807 15:14:45.557112 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.557116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.558094 (+ 978us) server_negotiation.cc:407] Connection header received
0807 15:14:45.559447 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.559450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.559509 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.559591 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.560768 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.561320 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.562502 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.562734 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.562850 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.565314 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.565320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.565325 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.565357 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.567260 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.567896 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.567900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.567902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.568014 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.568371 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.568373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.568375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.568574 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.568659 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.569114 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.569218 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:14:45.570304 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.557363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33695 (local address 127.31.230.195:55149)
0807 15:14:45.557951 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.557993 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.559229 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.559771 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.559778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.560138 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:45.560613 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.560626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.561451 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.561455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.562369 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.562378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.562893 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.562897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.563075 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.563653 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:45.563669 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:45.565115 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:45.567384 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:45.567390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:45.567405 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:45.567765 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:45.568127 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:45.568130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:45.568132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:45.568278 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:45.568667 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:45.568672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:45.568888 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.569042 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.569310 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":186,"threads_started":1}
I20260807 15:14:45.571453 1692 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33695
I20260807 15:14:45.571736 1692 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:45.572240 1692 heartbeater.cc:511] Master 127.31.230.254:33695 requested a full tablet report, sending...
I20260807 15:14:45.573257 1023 ts_manager.cc:194] Registered new tserver with Master: 55aa6e6d88974a7e9077bf2c84f91413 (127.31.230.195:43985)
I20260807 15:14:45.573899 1023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:55149
I20260807 15:14:45.579865 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:45.590361 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.583517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.31.230.254:33695)
0807 15:14:45.583646 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.583649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.583835 (+ 186us) server_negotiation.cc:407] Connection header received
0807 15:14:45.583932 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.583934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.583973 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.584032 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.584687 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.585203 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.586049 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.586274 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.586355 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.587619 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.587622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.587624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.587646 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.588847 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.589244 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.589247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.589248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.589322 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.589581 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.589584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.589586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.589842 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.589947 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.590154 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.590247 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:14:45.594817 1023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:45.597700 1023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:45.612390 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.606136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:41057 (local address 127.0.0.1:59152)
0807 15:14:45.606928 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.606940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.607022 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.607818 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.607821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.607835 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:45.607937 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.607944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.609335 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.609338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.610921 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.610933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.611807 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.611811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.611928 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.611989 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.612095 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.612237 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":79,"thread_start_us":95,"threads_started":1}
I20260807 15:14:45.612533 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.607348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.31.230.194:41165)
0807 15:14:45.607653 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.607665 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.608265 (+ 600us) server_negotiation.cc:407] Connection header received
0807 15:14:45.608354 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.608359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.608545 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.608687 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:45.608941 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.609866 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.611283 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.611898 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.612021 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.612153 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.612276 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.612365 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":128,"threads_started":1}
I20260807 15:14:45.612453 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.607145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41165 (local address 127.0.0.1:60520)
0807 15:14:45.608137 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.608150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.608251 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.608726 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.608729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.608748 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:45.608825 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.608830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.609996 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.610001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.611156 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.611167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.612026 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.612030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.612134 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.612159 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.612245 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.612312 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":32,"thread_start_us":91,"threads_started":1}
I20260807 15:14:45.613060 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.606725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59152 (local address 127.31.230.193:41057)
0807 15:14:45.607156 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.607166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.607194 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:14:45.607275 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.607280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.607489 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.607666 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:45.608084 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.609194 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.611054 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.611680 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.611783 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.611879 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.612789 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.612905 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":125,"threads_started":1}
I20260807 15:14:45.613276 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.606014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:43985 (local address 127.0.0.1:54662)
0807 15:14:45.606502 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.606541 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.606678 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.607671 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.607677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.607700 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:45.607831 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.607838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.609295 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.609298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.610943 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.610949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.612898 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.612902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.612989 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.613009 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.613077 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.613150 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":192,"thread_start_us":89,"threads_started":1}
I20260807 15:14:45.613598 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.606140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.31.230.195:43985)
0807 15:14:45.607155 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.607173 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.607193 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:14:45.607274 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.607280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.607489 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.607992 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:45.608180 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.609160 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.611930 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.612778 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.612913 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.613038 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.613098 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.613196 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":110,"threads_started":1}
I20260807 15:14:45.615875 1211 tablet_service.cc:1467] Processing CreateTablet for tablet 4314b9b886f64b898bdee17db092dda6 (DEFAULT_TABLE table=test-table [id=4494a26a829a47fb94e06c65839d5912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:45.615936 1415 tablet_service.cc:1467] Processing CreateTablet for tablet 4314b9b886f64b898bdee17db092dda6 (DEFAULT_TABLE table=test-table [id=4494a26a829a47fb94e06c65839d5912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:45.616146 1598 tablet_service.cc:1467] Processing CreateTablet for tablet 4314b9b886f64b898bdee17db092dda6 (DEFAULT_TABLE table=test-table [id=4494a26a829a47fb94e06c65839d5912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:45.616911 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4314b9b886f64b898bdee17db092dda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:45.616982 1415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4314b9b886f64b898bdee17db092dda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:45.616997 1598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4314b9b886f64b898bdee17db092dda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:45.623906 1713 tablet_bootstrap.cc:492] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: Bootstrap starting.
I20260807 15:14:45.624814 1715 tablet_bootstrap.cc:492] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: Bootstrap starting.
I20260807 15:14:45.625916 1714 tablet_bootstrap.cc:492] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: Bootstrap starting.
I20260807 15:14:45.626511 1713 tablet_bootstrap.cc:654] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:45.627004 1715 tablet_bootstrap.cc:654] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:45.627332 1713 log.cc:824] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:45.628358 1715 log.cc:824] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:45.628780 1714 tablet_bootstrap.cc:654] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:45.629671 1714 log.cc:824] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:45.629710 1713 tablet_bootstrap.cc:492] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: No bootstrap required, opened a new log
I20260807 15:14:45.629921 1713 ts_tablet_manager.cc:1397] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:14:45.630447 1715 tablet_bootstrap.cc:492] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: No bootstrap required, opened a new log
I20260807 15:14:45.630642 1715 ts_tablet_manager.cc:1397] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 15:14:45.633370 1713 raft_consensus.cc:348] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.633370 1715 raft_consensus.cc:348] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.633663 1713 raft_consensus.cc:374] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:45.633665 1715 raft_consensus.cc:374] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:45.633721 1715 raft_consensus.cc:729] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55aa6e6d88974a7e9077bf2c84f91413, State: Initialized, Role: FOLLOWER
I20260807 15:14:45.633739 1713 raft_consensus.cc:729] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff6a87af1734061b15f1bbfd8538d8b, State: Initialized, Role: FOLLOWER
I20260807 15:14:45.634181 1715 consensus_queue.cc:260] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.634212 1713 consensus_queue.cc:260] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.634856 1692 heartbeater.cc:503] Master 127.31.230.254:33695 was elected leader, sending a full tablet report...
I20260807 15:14:45.635185 1713 ts_tablet_manager.cc:1428] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:14:45.635195 1715 ts_tablet_manager.cc:1428] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:45.635324 1714 tablet_bootstrap.cc:492] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: No bootstrap required, opened a new log
I20260807 15:14:45.635506 1714 ts_tablet_manager.cc:1397] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260807 15:14:45.636019 1313 heartbeater.cc:503] Master 127.31.230.254:33695 was elected leader, sending a full tablet report...
I20260807 15:14:45.639290 1714 raft_consensus.cc:348] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.639503 1714 raft_consensus.cc:374] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:45.639537 1714 raft_consensus.cc:729] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84d0c08515642a88473cff565419804, State: Initialized, Role: FOLLOWER
I20260807 15:14:45.640018 1714 consensus_queue.cc:260] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.640672 1507 heartbeater.cc:503] Master 127.31.230.254:33695 was elected leader, sending a full tablet report...
I20260807 15:14:45.640980 1714 ts_tablet_manager.cc:1428] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:14:45.791867 1724 raft_consensus.cc:482] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:45.792096 1724 raft_consensus.cc:504] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.793320 1724 leader_election.cc:288] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff6a87af1734061b15f1bbfd8538d8b (127.31.230.193:41057), d84d0c08515642a88473cff565419804 (127.31.230.194:41165)
I20260807 15:14:45.797667 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.793610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41165 (local address 127.31.230.195:56831)
0807 15:14:45.793729 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.793746 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.793853 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.794151 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.794155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.794178 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:45.794319 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.794325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.795301 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.795306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.796439 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.796450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.797271 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.797273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.797356 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.797377 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.797439 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.797494 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:14:45.797720 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.793740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:56831 (local address 127.31.230.194:41165)
0807 15:14:45.793879 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.793883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.793903 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:14:45.793968 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.793971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.794024 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.794128 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:45.794445 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.795172 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.796576 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.797151 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.797278 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.797332 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.797479 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.797580 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:14:45.798400 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.793609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:41057 (local address 127.31.230.195:44373)
0807 15:14:45.794149 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:45.794162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:45.794326 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:45.794666 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:45.794672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:45.794694 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:45.794824 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.794830 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.795733 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.795736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.796871 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:45.796881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.797937 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.797942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:45.798043 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.798069 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:14:45.798184 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:14:45.798258 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":151,"thread_start_us":119,"threads_started":1}
I20260807 15:14:45.798419 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.793712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:44373 (local address 127.31.230.193:41057)
0807 15:14:45.793840 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.793845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.794259 (+ 414us) server_negotiation.cc:407] Connection header received
0807 15:14:45.794459 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.794462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.794515 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.794598 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:45.794991 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.795616 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.796999 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.797670 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.797792 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.797842 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.798200 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.798275 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:14:45.798338 1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4314b9b886f64b898bdee17db092dda6" candidate_uuid: "55aa6e6d88974a7e9077bf2c84f91413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d84d0c08515642a88473cff565419804" is_pre_election: true
I20260807 15:14:45.798674 1443 raft_consensus.cc:2454] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55aa6e6d88974a7e9077bf2c84f91413 in term 0.
I20260807 15:14:45.799238 1539 leader_election.cc:302] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55aa6e6d88974a7e9077bf2c84f91413, d84d0c08515642a88473cff565419804; no voters:
I20260807 15:14:45.799479 1242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4314b9b886f64b898bdee17db092dda6" candidate_uuid: "55aa6e6d88974a7e9077bf2c84f91413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" is_pre_election: true
I20260807 15:14:45.799546 1724 raft_consensus.cc:2781] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:45.799639 1724 raft_consensus.cc:482] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:45.799685 1724 raft_consensus.cc:3037] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:45.799736 1242 raft_consensus.cc:2454] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55aa6e6d88974a7e9077bf2c84f91413 in term 0.
I20260807 15:14:45.801011 1724 raft_consensus.cc:504] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
I20260807 15:14:45.801363 1724 leader_election.cc:288] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff6a87af1734061b15f1bbfd8538d8b (127.31.230.193:41057), d84d0c08515642a88473cff565419804 (127.31.230.194:41165)
I20260807 15:14:45.801718 1242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4314b9b886f64b898bdee17db092dda6" candidate_uuid: "55aa6e6d88974a7e9077bf2c84f91413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff6a87af1734061b15f1bbfd8538d8b"
I20260807 15:14:45.801847 1242 raft_consensus.cc:3037] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:45.802129 1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4314b9b886f64b898bdee17db092dda6" candidate_uuid: "55aa6e6d88974a7e9077bf2c84f91413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d84d0c08515642a88473cff565419804"
I20260807 15:14:45.802269 1443 raft_consensus.cc:3037] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:45.803130 1242 raft_consensus.cc:2454] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55aa6e6d88974a7e9077bf2c84f91413 in term 1.
I20260807 15:14:45.803463 1540 leader_election.cc:302] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff6a87af1734061b15f1bbfd8538d8b, 55aa6e6d88974a7e9077bf2c84f91413; no voters:
I20260807 15:14:45.803503 1443 raft_consensus.cc:2454] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55aa6e6d88974a7e9077bf2c84f91413 in term 1.
I20260807 15:14:45.803668 1724 raft_consensus.cc:2781] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:45.803918 1724 raft_consensus.cc:686] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [term 1 LEADER]: Becoming Leader. State: Replica: 55aa6e6d88974a7e9077bf2c84f91413, State: Running, Role: LEADER
I20260807 15:14:45.804252 1724 consensus_queue.cc:237] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } }
W20260807 15:14:45.806022 1693 tablet.cc:2369] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:45.807572 1023 catalog_manager.cc:5495] T 4314b9b886f64b898bdee17db092dda6 P 55aa6e6d88974a7e9077bf2c84f91413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55aa6e6d88974a7e9077bf2c84f91413 (127.31.230.195). New cstate: current_term: 1 leader_uuid: "55aa6e6d88974a7e9077bf2c84f91413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55aa6e6d88974a7e9077bf2c84f91413" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 43985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ff6a87af1734061b15f1bbfd8538d8b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 41057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d84d0c08515642a88473cff565419804" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41165 } health_report { overall_health: UNKNOWN } } }
W20260807 15:14:45.826319 1508 tablet.cc:2369] T 4314b9b886f64b898bdee17db092dda6 P d84d0c08515642a88473cff565419804: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:45.839920 1315 tablet.cc:2369] T 4314b9b886f64b898bdee17db092dda6 P 2ff6a87af1734061b15f1bbfd8538d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:14:45.858008 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.843853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.31.230.193:41057)
0807 15:14:45.844003 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.844010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.844182 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:14:45.844364 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.844367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.844410 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.844497 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.845192 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.845778 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.846624 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.846853 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.846964 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.854743 (+ 7779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.854757 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.854772 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.854797 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.856323 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.856719 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.856725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.856726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.856808 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.857036 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.857038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.857039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.857312 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.857468 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.857683 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.857787 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:14:45.886408 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.877669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.31.230.193:41057)
0807 15:14:45.877817 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.877822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.877978 (+ 156us) server_negotiation.cc:407] Connection header received
0807 15:14:45.878158 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.878161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.878201 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.878298 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.878972 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.879513 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.880430 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.880658 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.880793 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.883241 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.883246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.883250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.883278 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.884601 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.885019 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.885024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.885026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.885110 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.885401 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.885405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.885406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.885639 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.885784 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.886058 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.886184 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:14:45 dist-test-slave-gkw7 krb5kdc[955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:14:45.912891 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:45.904792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.31.230.193:41057)
0807 15:14:45.904959 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:45.904965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:45.905133 (+ 168us) server_negotiation.cc:407] Connection header received
0807 15:14:45.905246 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:45.905249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:45.905291 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:45.905361 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:45.906067 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.906560 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.907360 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:45.907571 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:45.907647 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:45.910078 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:45.910081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:45.910083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:45.910107 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:45.911553 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.911916 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.911918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.911920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.911994 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:45.912213 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:45.912216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:45.912217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:45.912368 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:45.912462 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:45.912671 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 15:14:45.912769 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 15:14:45.914364 1181 tablet_service.cc:2908] Rejecting scan request for tablet 4314b9b886f64b898bdee17db092dda6: Uninitialized: clean time has not yet been initialized
I20260807 15:14:45.916495 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 1110
I20260807 15:14:45.931409 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 1319
I20260807 15:14:45.941430 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 1515
I20260807 15:14:45.953109 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 973
2026-08-07T15:14:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2710 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:46 dist-test-slave-gkw7 krb5kdc[1749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:46 dist-test-slave-gkw7 krb5kdc[1749](info): set up 2 sockets
Aug 07 15:14:46 dist-test-slave-gkw7 krb5kdc[1749](info): commencing operation
krb5kdc: starting...
W20260807 15:14:47.208621 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:14:47 dist-test-slave-gkw7 krb5kdc[1749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:14:47.352864 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:43165
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:43165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:47.467756 1765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:47.468015 1765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:47.468055 1765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:47.472030 1765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:47.472093 1765 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:14:47.472110 1765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:47.472128 1765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:47.472144 1765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:47.472160 1765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:47.477270 1765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:43165
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:43165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:47.478390 1765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:47.479382 1765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:47.488180 1774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:47.488133 1771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:47.488133 1772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:47.488600 1765 server_base.cc:1034] running on GCE node
I20260807 15:14:47.489195 1765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:47.490335 1765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:47.491498 1765 hybrid_clock.cc:648] HybridClock initialized: now 1786115687491486 us; error 30 us; skew 500 ppm
Aug 07 15:14:47 dist-test-slave-gkw7 krb5kdc[1749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:47.494449 1765 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:14:47.495373 1765 webserver.cc:460] Webserver started at http://127.31.230.254:44521/ using document root <none> and password file <none>
I20260807 15:14:47.495937 1765 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:47.496030 1765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:47.496287 1765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:47.498446 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "a35e276231234913b30fb55876aaf691"
format_stamp: "Formatted at 2026-08-07 15:14:47 on dist-test-slave-gkw7"
server_key: "6602d119932720c69d4dc3d9773940b5"
server_key_iv: "2e252baae8bf20fbf27994f53ad31cd8"
server_key_version: "encryptionkey@0"
I20260807 15:14:47.499087 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "a35e276231234913b30fb55876aaf691"
format_stamp: "Formatted at 2026-08-07 15:14:47 on dist-test-slave-gkw7"
server_key: "6602d119932720c69d4dc3d9773940b5"
server_key_iv: "2e252baae8bf20fbf27994f53ad31cd8"
server_key_version: "encryptionkey@0"
I20260807 15:14:47.502872 1765 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:14:47.505335 1785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:47.506634 1765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:14:47.506785 1765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "a35e276231234913b30fb55876aaf691"
format_stamp: "Formatted at 2026-08-07 15:14:47 on dist-test-slave-gkw7"
server_key: "6602d119932720c69d4dc3d9773940b5"
server_key_iv: "2e252baae8bf20fbf27994f53ad31cd8"
server_key_version: "encryptionkey@0"
I20260807 15:14:47.506899 1765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:47.533552 1765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:47.537112 1765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:47.537335 1765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:47.546201 1765 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:43165
I20260807 15:14:47.546234 1873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:43165 every 8 connection(s)
I20260807 15:14:47.547183 1765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:47.549948 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 1765
I20260807 15:14:47.550040 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:14:47.550326 32667 external_mini_cluster.cc:1428] Setting key 4c28fb33b90d0aecb767e9f35d136a9f
I20260807 15:14:47.550382 1874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:47.557438 1874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: Bootstrap starting.
Aug 07 15:14:47 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:47.560003 1874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:47.561496 1874 log.cc:824] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:47.563694 1874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: No bootstrap required, opened a new log
I20260807 15:14:47.564941 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:47.551802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.31.230.254:43165)
0807 15:14:47.552298 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:47.552313 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:47.552351 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:14:47.553324 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:47.553343 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:47.553733 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:47.554076 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:47.554882 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.555877 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.556802 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.557280 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.557454 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:47.560002 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:47.560007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:47.560021 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:47.560049 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:47.562138 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:47.562602 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:47.562615 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:47.562620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:47.562734 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:47.563021 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:47.563026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:47.563027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:47.563429 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:47.563639 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:47.563941 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 15:14:47.564232 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":168,"threads_started":1}
I20260807 15:14:47.567751 1874 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e276231234913b30fb55876aaf691" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43165 } }
I20260807 15:14:47.568009 1874 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:47.568104 1874 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35e276231234913b30fb55876aaf691, State: Initialized, Role: FOLLOWER
I20260807 15:14:47.568557 1874 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e276231234913b30fb55876aaf691" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43165 } }
I20260807 15:14:47.568724 1874 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:47.568820 1874 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:47.568956 1874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:47.570070 1874 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e276231234913b30fb55876aaf691" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43165 } }
I20260807 15:14:47.570446 1874 leader_election.cc:302] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35e276231234913b30fb55876aaf691; no voters:
I20260807 15:14:47.570811 1874 leader_election.cc:288] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:47.570897 1881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:47.571192 1881 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [term 1 LEADER]: Becoming Leader. State: Replica: a35e276231234913b30fb55876aaf691, State: Running, Role: LEADER
I20260807 15:14:47.571511 1881 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e276231234913b30fb55876aaf691" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43165 } }
I20260807 15:14:47.572158 1874 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:47.573256 1885 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35e276231234913b30fb55876aaf691. Latest consensus state: current_term: 1 leader_uuid: "a35e276231234913b30fb55876aaf691" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e276231234913b30fb55876aaf691" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43165 } } }
I20260807 15:14:47.573333 1882 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35e276231234913b30fb55876aaf691" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e276231234913b30fb55876aaf691" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43165 } } }
I20260807 15:14:47.573428 1885 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:47.573431 1882 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:47.573890 1892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:47.576738 1892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:47.582831 1892 catalog_manager.cc:1347] Generated new cluster ID: 86f944a3516945c898da89df8c3f303b
I20260807 15:14:47.582892 1892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:47.593906 1892 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:47.595258 1892 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:47.610303 1892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: Generated new TSK 0
I20260807 15:14:47.611230 1892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:14:47.648013 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43165
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:47.768580 1912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:47.768836 1912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:47.768877 1912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:47.772946 1912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:47.773006 1912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:47.773084 1912 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:14:47.773120 1912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:14:47.778479 1912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:43165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:47.779450 1912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:47.780496 1912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:47.790437 1918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:47.790458 1920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:47.790483 1917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:47.790777 1912 server_base.cc:1034] running on GCE node
I20260807 15:14:47.791148 1912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:47.791729 1912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:47.792891 1912 hybrid_clock.cc:648] HybridClock initialized: now 1786115687792875 us; error 27 us; skew 500 ppm
Aug 07 15:14:47 dist-test-slave-gkw7 krb5kdc[1749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:47.795727 1912 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:14:47.796651 1912 webserver.cc:460] Webserver started at http://127.31.230.193:35743/ using document root <none> and password file <none>
I20260807 15:14:47.797235 1912 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:47.797346 1912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:47.797649 1912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:47.799851 1912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "d67bfaa572ee40eab99fc3a23d368f62"
format_stamp: "Formatted at 2026-08-07 15:14:47 on dist-test-slave-gkw7"
server_key: "ce65bc031fcdb25cc021a78a40ea9715"
server_key_iv: "aefde6b2e9871195b756c67c933b8086"
server_key_version: "encryptionkey@0"
I20260807 15:14:47.800477 1912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "d67bfaa572ee40eab99fc3a23d368f62"
format_stamp: "Formatted at 2026-08-07 15:14:47 on dist-test-slave-gkw7"
server_key: "ce65bc031fcdb25cc021a78a40ea9715"
server_key_iv: "aefde6b2e9871195b756c67c933b8086"
server_key_version: "encryptionkey@0"
I20260807 15:14:47.804469 1912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:14:47.807227 1930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:47.808575 1912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:14:47.808727 1912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "d67bfaa572ee40eab99fc3a23d368f62"
format_stamp: "Formatted at 2026-08-07 15:14:47 on dist-test-slave-gkw7"
server_key: "ce65bc031fcdb25cc021a78a40ea9715"
server_key_iv: "aefde6b2e9871195b756c67c933b8086"
server_key_version: "encryptionkey@0"
I20260807 15:14:47.808854 1912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:47.840265 1912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:47.843633 1912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:47.843853 1912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:47.845469 1912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:47.845517 1912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:47.845624 1912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:47.845660 1912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:14:47 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:47.857604 1912 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:36239
I20260807 15:14:47.857745 2108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:36239 every 8 connection(s)
I20260807 15:14:47.858664 1912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:47.860586 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:47.846563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:34567 (local address 127.31.230.254:43165)
0807 15:14:47.846708 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:47.846713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:47.847844 (+ 1131us) server_negotiation.cc:407] Connection header received
0807 15:14:47.848976 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:47.848979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:47.849040 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:47.849138 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:47.850798 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.851427 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.852660 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.852996 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.853136 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:47.856534 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:47.856538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:47.856541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:47.856577 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:47.858181 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:47.858781 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:47.858785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:47.858787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:47.858879 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:47.859353 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:47.859356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:47.859358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:47.859612 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:47.859720 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:47.860327 (+ 607us) server_negotiation.cc:299] Negotiation successful
0807 15:14:47.860451 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:14:47.861423 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:47.847091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.31.230.193:34567)
0807 15:14:47.847672 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:47.847716 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:47.848720 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:47.849354 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:47.849365 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:47.849892 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:47.850607 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:47.850630 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.851576 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.851580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:47.852506 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:47.852516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.853198 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.853202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:47.853375 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:47.854051 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:47.854072 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:47.856348 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:47.858329 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:47.858335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:47.858351 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:47.858638 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:47.859019 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:47.859022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:47.859024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:47.859234 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:47.859733 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:47.859738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:47.859989 (+ 251us) client_negotiation.cc:769] Sending connection context
0807 15:14:47.860215 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 15:14:47.860507 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":140,"threads_started":1}
I20260807 15:14:47.863149 2109 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43165
I20260807 15:14:47.863335 2109 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:47.863884 2109 heartbeater.cc:511] Master 127.31.230.254:43165 requested a full tablet report, sending...
I20260807 15:14:47.865571 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 1912
I20260807 15:14:47.865712 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:47.865813 1809 ts_manager.cc:194] Registered new tserver with Master: d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239)
I20260807 15:14:47.866020 32667 external_mini_cluster.cc:1428] Setting key e44f962935e79876ea0b8da06ac0bd3f
I20260807 15:14:47.867157 1809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:34567
I20260807 15:14:47.875342 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:47.865489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.31.230.193:44159)
0807 15:14:47.865870 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:47.865884 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:47.866018 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:47.866311 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:47.866314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:47.866539 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:47.866644 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:47.866650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.867472 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.867475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:47.869666 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:47.869676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.870167 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.870171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:47.870271 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:47.870914 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:47.870936 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:47.871349 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:47.873691 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:47.873694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:47.873695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:47.873889 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:47.874265 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:47.874267 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:47.874269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:47.874407 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:47.874910 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:47.874912 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:47.874990 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 15:14:47.875067 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:14:47.875176 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":132,"threads_started":1}
I20260807 15:14:47.875581 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:47.865581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:44159 (local address 127.31.230.254:43165)
0807 15:14:47.865716 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:47.865722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:47.865966 (+ 244us) server_negotiation.cc:407] Connection header received
0807 15:14:47.866139 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:47.866143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:47.866198 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:47.866294 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:47.866772 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.867312 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.869796 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:47.870039 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:47.870164 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:47.871508 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:47.871515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:47.871520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:47.871552 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:47.873578 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:47.874038 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:47.874045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:47.874050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:47.874165 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:47.874513 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:47.874519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:47.874520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:47.874797 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:47.874913 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:47.875173 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 15:14:47.875336 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:14:47.901597 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43165
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:48.020790 2121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:48.021087 2121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:48.021170 2121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:48.025362 2121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:48.025442 2121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:48.025522 2121 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:14:48.025633 2121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:14:48.031181 2121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:43165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:48.032398 2121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:48.033564 2121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:48.043615 2128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:48.043625 2129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:48.043644 2131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:48.043812 2121 server_base.cc:1034] running on GCE node
I20260807 15:14:48.044396 2121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:48.045079 2121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:48.046275 2121 hybrid_clock.cc:648] HybridClock initialized: now 1786115688046259 us; error 29 us; skew 500 ppm
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:48.049129 2121 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:14:48.050091 2121 webserver.cc:460] Webserver started at http://127.31.230.194:41883/ using document root <none> and password file <none>
I20260807 15:14:48.050678 2121 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:48.050776 2121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:48.051019 2121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:48.053120 2121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "55799da85e05420eb8c6b82305232453"
format_stamp: "Formatted at 2026-08-07 15:14:48 on dist-test-slave-gkw7"
server_key: "1e831c8dab7b3d2b4efd6ed83fbecd73"
server_key_iv: "004fd41ad9b6e01598195040039a2b50"
server_key_version: "encryptionkey@0"
I20260807 15:14:48.053782 2121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "55799da85e05420eb8c6b82305232453"
format_stamp: "Formatted at 2026-08-07 15:14:48 on dist-test-slave-gkw7"
server_key: "1e831c8dab7b3d2b4efd6ed83fbecd73"
server_key_iv: "004fd41ad9b6e01598195040039a2b50"
server_key_version: "encryptionkey@0"
I20260807 15:14:48.057698 2121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:14:48.060253 2141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:48.061486 2121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:48.061664 2121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "55799da85e05420eb8c6b82305232453"
format_stamp: "Formatted at 2026-08-07 15:14:48 on dist-test-slave-gkw7"
server_key: "1e831c8dab7b3d2b4efd6ed83fbecd73"
server_key_iv: "004fd41ad9b6e01598195040039a2b50"
server_key_version: "encryptionkey@0"
I20260807 15:14:48.061764 2121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:48.083027 2121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:48.086469 2121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:48.086624 2121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:48.088024 2121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:48.088073 2121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:48.088116 2121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:48.088183 2121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:48.100195 2121 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:41729
I20260807 15:14:48.100286 2306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:41729 every 8 connection(s)
I20260807 15:14:48.101284 2121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:48.103470 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.089059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:60199 (local address 127.31.230.254:43165)
0807 15:14:48.089279 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.089284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.090465 (+ 1181us) server_negotiation.cc:407] Connection header received
0807 15:14:48.091685 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.091689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.091751 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.091851 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.093610 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.094425 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.095625 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.095969 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.096098 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.099379 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.099382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.099385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.099456 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.101159 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.101821 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.101825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.101827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.101928 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.102404 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.102407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.102408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.102603 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.102688 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.103215 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.103341 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 15:14:48.104254 2159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.089768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.31.230.194:60199)
0807 15:14:48.090296 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.090342 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.091449 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.092091 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.092102 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.092580 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:48.093386 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.093412 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.094554 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.094558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.095489 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.095499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.096167 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.096172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.096408 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.097186 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:48.097204 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:48.099183 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:48.101280 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.101285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.101301 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.101669 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.102049 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.102051 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.102053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.102266 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.102711 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:48.102716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:48.102939 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.103101 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.103360 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":130,"threads_started":1}
I20260807 15:14:48.105737 2307 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43165
I20260807 15:14:48.105904 2307 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:48.106417 2307 heartbeater.cc:511] Master 127.31.230.254:43165 requested a full tablet report, sending...
I20260807 15:14:48.107537 1809 ts_manager.cc:194] Registered new tserver with Master: 55799da85e05420eb8c6b82305232453 (127.31.230.194:41729)
I20260807 15:14:48.108351 1809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:60199
I20260807 15:14:48.109982 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 2121
I20260807 15:14:48.110110 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:48.110378 32667 external_mini_cluster.cc:1428] Setting key 34a936a78151170164d744f21594e759
I20260807 15:14:48.117259 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.108052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:44921 (local address 127.31.230.254:43165)
0807 15:14:48.108188 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.108192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.108363 (+ 171us) server_negotiation.cc:407] Connection header received
0807 15:14:48.108438 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.108441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.108481 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.108568 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.109122 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.110915 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.111871 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.112127 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.112222 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.113756 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.113760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.113762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.113792 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.115503 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.115938 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.115940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.115942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.116041 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.116383 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.116385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.116386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.116607 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.116686 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.117025 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.117124 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:14:48.117501 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.107926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.31.230.194:44921)
0807 15:14:48.108279 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.108291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.108373 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.108607 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.108610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.108849 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:48.108995 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.109003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.111043 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.111047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.111736 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.111748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.112327 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.112335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.112460 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.112987 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:48.113006 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:48.113578 (+ 572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:48.115634 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.115637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.115638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.115848 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.116162 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.116168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.116173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.116284 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.116715 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:48.116721 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:48.116846 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.116969 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.117210 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:14:48.140725 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43165
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:48.256536 2319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:48.256850 2319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:48.256943 2319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:48.260890 2319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:48.260947 2319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:48.260985 2319 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:14:48.261016 2319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:14:48.266433 2319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:43165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:48.267544 2319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:48.268541 2319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:48.278421 2327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:48.278438 2328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:48.278456 2319 server_base.cc:1034] running on GCE node
W20260807 15:14:48.278398 2330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:48.279091 2319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:48.279697 2319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:48.280851 2319 hybrid_clock.cc:648] HybridClock initialized: now 1786115688280811 us; error 51 us; skew 500 ppm
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:48.283578 2319 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:14:48.284447 2319 webserver.cc:460] Webserver started at http://127.31.230.195:36215/ using document root <none> and password file <none>
I20260807 15:14:48.284996 2319 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:48.285089 2319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:48.285360 2319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:48.287492 2319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "31d7e1eff31c41a3a5ae57f12b30335b"
format_stamp: "Formatted at 2026-08-07 15:14:48 on dist-test-slave-gkw7"
server_key: "80d8fdd8f92b328dcfc121b53ff347cf"
server_key_iv: "ccec1057be75b437b207386dccf01693"
server_key_version: "encryptionkey@0"
I20260807 15:14:48.288082 2319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "31d7e1eff31c41a3a5ae57f12b30335b"
format_stamp: "Formatted at 2026-08-07 15:14:48 on dist-test-slave-gkw7"
server_key: "80d8fdd8f92b328dcfc121b53ff347cf"
server_key_iv: "ccec1057be75b437b207386dccf01693"
server_key_version: "encryptionkey@0"
I20260807 15:14:48.291872 2319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 15:14:48.294318 2337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:48.295499 2319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:14:48.295635 2319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "31d7e1eff31c41a3a5ae57f12b30335b"
format_stamp: "Formatted at 2026-08-07 15:14:48 on dist-test-slave-gkw7"
server_key: "80d8fdd8f92b328dcfc121b53ff347cf"
server_key_iv: "ccec1057be75b437b207386dccf01693"
server_key_version: "encryptionkey@0"
I20260807 15:14:48.295755 2319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:48.316138 2319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:48.319244 2319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:48.319453 2319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:48.320976 2319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:48.321048 2319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:48.321130 2319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:48.321172 2319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:48.332722 2319 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:36785
I20260807 15:14:48.332806 2507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:36785 every 8 connection(s)
I20260807 15:14:48.333909 2319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:48.335284 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.321719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:37189 (local address 127.31.230.254:43165)
0807 15:14:48.321899 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.321905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.323298 (+ 1393us) server_negotiation.cc:407] Connection header received
0807 15:14:48.324485 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.324488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.324546 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.324628 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.326105 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.326665 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.327992 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.328268 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.328390 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.331290 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.331300 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.331303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.331328 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.333129 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.333652 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.333655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.333657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.333751 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.334099 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.334102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.334104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.334315 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.334428 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.335050 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.335166 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:14:48.336040 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.322669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.31.230.195:37189)
0807 15:14:48.323141 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.323198 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.324228 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.324806 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.324813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.325208 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:48.325961 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.325978 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.326800 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.326803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.327870 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.327880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.328439 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.328444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.328620 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.329294 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:48.329313 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:48.331114 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:48.333241 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.333246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.333257 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.333499 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.333853 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.333856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.333857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.333988 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.334425 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:48.334432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:48.334759 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.334953 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.335224 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":157,"threads_started":1}
I20260807 15:14:48.337297 2508 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43165
I20260807 15:14:48.337483 2508 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:48.338063 2508 heartbeater.cc:511] Master 127.31.230.254:43165 requested a full tablet report, sending...
I20260807 15:14:48.338413 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 2319
I20260807 15:14:48.338543 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:48.338837 32667 external_mini_cluster.cc:1428] Setting key aaf2d7f2d30118a7e5eb0b9f15d96de5
I20260807 15:14:48.339342 1808 ts_manager.cc:194] Registered new tserver with Master: 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785)
I20260807 15:14:48.340258 1808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:37189
I20260807 15:14:48.341081 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:48.347929 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.339591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.31.230.195:38427)
0807 15:14:48.340037 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.340049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.340172 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.340428 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.340431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.340665 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:48.340761 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.340766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.341829 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.341835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.342836 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.342851 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.343340 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.343346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.343473 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.344078 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:48.344089 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:48.344400 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:48.346333 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.346339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.346344 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.346620 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.346922 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.346926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.346928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.347029 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.347477 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:48.347484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:48.347589 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.347676 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.347791 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":116,"threads_started":1}
I20260807 15:14:48.348150 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.339675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:38427 (local address 127.31.230.254:43165)
0807 15:14:48.339790 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.339795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.340127 (+ 332us) server_negotiation.cc:407] Connection header received
0807 15:14:48.340280 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.340284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.340325 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.340414 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.340872 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.341641 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.342963 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.343203 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.343295 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.344544 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.344550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.344557 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.344578 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.346217 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.346732 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.346735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.346736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.346819 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.347139 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.347143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.347144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.347325 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.347455 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.347779 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.347912 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:48.370764 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.361739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55874 (local address 127.31.230.254:43165)
0807 15:14:48.361880 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.361884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.361897 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:14:48.361939 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.361941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.361983 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.362087 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.362902 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.363542 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.364503 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.364763 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.364888 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.367601 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.367604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.367607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.367634 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.368907 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.369378 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.369382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.369385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.369472 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.369836 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.369840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.369844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.370031 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.370186 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.370450 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.370587 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:14:48.373958 1809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:48.377126 1809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:48.392042 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.386133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.31.230.193:36239)
0807 15:14:48.386497 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.386506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.386530 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:14:48.386610 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.386617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.386869 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.387396 (+ 527us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.387531 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.388696 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.390500 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.391268 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.391381 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.391480 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.391838 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.391926 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":62,"threads_started":1}
I20260807 15:14:48.392035 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.385989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.31.230.195:36785)
0807 15:14:48.386995 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.387010 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.387030 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:14:48.387100 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.387105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.387309 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.387442 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.387789 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.388845 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.390334 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.391122 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.391277 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.391428 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.391772 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.391889 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":123,"threads_started":1}
I20260807 15:14:48.392035 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.385850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36239 (local address 127.0.0.1:35692)
0807 15:14:48.386217 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.386283 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.386419 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.387039 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.387047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.387073 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.387226 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.387234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.388833 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.388837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.390376 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.390386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.391392 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.391396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.391532 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.391588 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.391709 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.391870 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":30,"thread_start_us":87,"threads_started":1}
I20260807 15:14:48.392597 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.385850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36785 (local address 127.0.0.1:48814)
0807 15:14:48.386653 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.386665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.386738 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.387442 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.387446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.387469 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.387608 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.387619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.388952 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.388956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.390210 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.390219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.391498 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.391503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.391605 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.391622 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.391716 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.392428 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":1081,"thread_start_us":92,"threads_started":1}
I20260807 15:14:48.393436 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.386679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41729 (local address 127.0.0.1:36034)
0807 15:14:48.387533 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.387550 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.387658 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.388289 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.388292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.388312 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.388416 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.388427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.390135 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.390139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.391062 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.391073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.392516 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.392520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.393176 (+ 656us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.393200 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.393281 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.393346 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260807 15:14:48.393851 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.387117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36034 (local address 127.31.230.194:41729)
0807 15:14:48.387903 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.387915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.387933 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:14:48.387995 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.387999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.388137 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.388307 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.388559 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.389992 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.391199 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.392119 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.392531 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.392924 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.393641 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.393738 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":133,"threads_started":1}
I20260807 15:14:48.395594 2009 tablet_service.cc:1467] Processing CreateTablet for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 (DEFAULT_TABLE table=test-table [id=43f74a505f144bc69e8194e3126f9c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:48.395605 2414 tablet_service.cc:1467] Processing CreateTablet for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 (DEFAULT_TABLE table=test-table [id=43f74a505f144bc69e8194e3126f9c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:48.396843 2009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.396850 2414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.396430 2214 tablet_service.cc:1467] Processing CreateTablet for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 (DEFAULT_TABLE table=test-table [id=43f74a505f144bc69e8194e3126f9c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:48.397445 2214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.404259 2543 tablet_bootstrap.cc:492] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Bootstrap starting.
I20260807 15:14:48.405977 2544 tablet_bootstrap.cc:492] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Bootstrap starting.
I20260807 15:14:48.406932 2543 tablet_bootstrap.cc:654] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.407846 2543 log.cc:824] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:48.408257 2544 tablet_bootstrap.cc:654] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.409001 2544 log.cc:824] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:48.409708 2546 tablet_bootstrap.cc:492] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Bootstrap starting.
I20260807 15:14:48.409880 2543 tablet_bootstrap.cc:492] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: No bootstrap required, opened a new log
I20260807 15:14:48.410097 2543 ts_tablet_manager.cc:1397] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 15:14:48.411170 2544 tablet_bootstrap.cc:492] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: No bootstrap required, opened a new log
I20260807 15:14:48.411362 2544 ts_tablet_manager.cc:1397] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:14:48.412127 2546 tablet_bootstrap.cc:654] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.413147 2546 log.cc:824] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:48.413800 2543 raft_consensus.cc:348] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.414100 2543 raft_consensus.cc:374] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.414170 2543 raft_consensus.cc:729] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55799da85e05420eb8c6b82305232453, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.414614 2543 consensus_queue.cc:260] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.415284 2307 heartbeater.cc:503] Master 127.31.230.254:43165 was elected leader, sending a full tablet report...
I20260807 15:14:48.415625 2543 ts_tablet_manager.cc:1428] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:14:48.416186 2546 tablet_bootstrap.cc:492] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: No bootstrap required, opened a new log
I20260807 15:14:48.416394 2546 ts_tablet_manager.cc:1397] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260807 15:14:48.416352 2544 raft_consensus.cc:348] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.416652 2544 raft_consensus.cc:374] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.416726 2544 raft_consensus.cc:729] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67bfaa572ee40eab99fc3a23d368f62, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.417171 2544 consensus_queue.cc:260] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.417860 2109 heartbeater.cc:503] Master 127.31.230.254:43165 was elected leader, sending a full tablet report...
I20260807 15:14:48.418200 2544 ts_tablet_manager.cc:1428] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 15:14:48.420887 2546 raft_consensus.cc:348] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.421164 2546 raft_consensus.cc:374] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.421239 2546 raft_consensus.cc:729] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d7e1eff31c41a3a5ae57f12b30335b, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.421685 2546 consensus_queue.cc:260] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.422351 2508 heartbeater.cc:503] Master 127.31.230.254:43165 was elected leader, sending a full tablet report...
I20260807 15:14:48.422564 2546 ts_tablet_manager.cc:1428] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 15:14:48.507201 2557 raft_consensus.cc:482] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:48.507402 2557 raft_consensus.cc:504] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.508553 2557 leader_election.cc:288] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239), 55799da85e05420eb8c6b82305232453 (127.31.230.194:41729)
I20260807 15:14:48.512542 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.508932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:48261 (local address 127.31.230.193:36239)
0807 15:14:48.509077 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.509082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.509119 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:14:48.509179 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.509182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.509232 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.509317 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.509644 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.510326 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.511490 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.511961 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.512106 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.512165 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.512344 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.512418 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:14:48.512542 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.508784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36239 (local address 127.31.230.195:48261)
0807 15:14:48.508932 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.508946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.509067 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.509338 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.509341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.509358 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.509499 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.509504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.510442 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.510447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.511363 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.511373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.512091 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.512095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.512208 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.512236 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.512328 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.512404 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:14:48.513245 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af4fdd9d8ea4be8a64e3b15fb60aa63" candidate_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67bfaa572ee40eab99fc3a23d368f62" is_pre_election: true
I20260807 15:14:48.513648 2036 raft_consensus.cc:2454] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d7e1eff31c41a3a5ae57f12b30335b in term 0.
I20260807 15:14:48.513764 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.509250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:60639 (local address 127.31.230.194:41729)
0807 15:14:48.509373 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.509377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.509692 (+ 315us) server_negotiation.cc:407] Connection header received
0807 15:14:48.509863 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.509866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.509917 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.510004 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.510287 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.510809 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.512584 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.513170 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.513321 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.513395 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.513571 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.513640 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:14:48.513815 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.509093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41729 (local address 127.31.230.195:60639)
0807 15:14:48.509584 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.509597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.509724 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.510018 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.510021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.510037 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.510170 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.510176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.510938 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.510942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.512464 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.512474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.513321 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.513325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.513429 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.513459 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.513552 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.513621 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":171,"threads_started":1}
I20260807 15:14:48.514210 2340 leader_election.cc:302] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d7e1eff31c41a3a5ae57f12b30335b, d67bfaa572ee40eab99fc3a23d368f62; no voters:
I20260807 15:14:48.514513 2557 raft_consensus.cc:2781] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:48.514515 2247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af4fdd9d8ea4be8a64e3b15fb60aa63" candidate_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55799da85e05420eb8c6b82305232453" is_pre_election: true
I20260807 15:14:48.514626 2557 raft_consensus.cc:482] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:48.514674 2557 raft_consensus.cc:3037] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.514797 2247 raft_consensus.cc:2454] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d7e1eff31c41a3a5ae57f12b30335b in term 0.
I20260807 15:14:48.515808 2557 raft_consensus.cc:504] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.516100 2557 leader_election.cc:288] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [CANDIDATE]: Term 1 election: Requested vote from peers d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239), 55799da85e05420eb8c6b82305232453 (127.31.230.194:41729)
I20260807 15:14:48.516546 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af4fdd9d8ea4be8a64e3b15fb60aa63" candidate_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67bfaa572ee40eab99fc3a23d368f62"
I20260807 15:14:48.516636 2247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af4fdd9d8ea4be8a64e3b15fb60aa63" candidate_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55799da85e05420eb8c6b82305232453"
I20260807 15:14:48.516731 2247 raft_consensus.cc:3037] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.516736 2036 raft_consensus.cc:3037] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.518047 2036 raft_consensus.cc:2454] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d7e1eff31c41a3a5ae57f12b30335b in term 1.
I20260807 15:14:48.518194 2247 raft_consensus.cc:2454] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d7e1eff31c41a3a5ae57f12b30335b in term 1.
I20260807 15:14:48.518400 2340 leader_election.cc:302] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d7e1eff31c41a3a5ae57f12b30335b, d67bfaa572ee40eab99fc3a23d368f62; no voters:
I20260807 15:14:48.518607 2557 raft_consensus.cc:2781] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:48.518885 2557 raft_consensus.cc:686] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 LEADER]: Becoming Leader. State: Replica: 31d7e1eff31c41a3a5ae57f12b30335b, State: Running, Role: LEADER
I20260807 15:14:48.519212 2557 consensus_queue.cc:237] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } }
I20260807 15:14:48.522584 1808 catalog_manager.cc:5495] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b reported cstate change: term changed from 0 to 1, leader changed from <none> to 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195). New cstate: current_term: 1 leader_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:48.551429 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.547073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.31.230.195:36785)
0807 15:14:48.547213 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.547216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.547232 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:14:48.547447 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.547450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.547504 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.547591 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:48.547929 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.548503 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.549812 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.550030 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.550121 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.550506 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:48.550929 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:48.551031 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.551162 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.551224 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:14:48.557997 2247 raft_consensus.cc:1261] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Refusing update from remote peer 31d7e1eff31c41a3a5ae57f12b30335b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:48.558000 2036 raft_consensus.cc:1261] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Refusing update from remote peer 31d7e1eff31c41a3a5ae57f12b30335b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:48.558770 2559 consensus_queue.cc:1035] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:48.559041 2557 consensus_queue.cc:1035] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [LEADER]: Connected to new peer: Peer: permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:48.566641 2562 mvcc.cc:205] Tried to move back new op lower bound from 7315929860325642240 to 7315929860177563648. Current Snapshot: MvccSnapshot[applied={T|T < 7315929860325642240}]
I20260807 15:14:48.569135 2563 mvcc.cc:205] Tried to move back new op lower bound from 7315929860325642240 to 7315929860177563648. Current Snapshot: MvccSnapshot[applied={T|T < 7315929860325642240}]
I20260807 15:14:48.581111 1808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:48.581275 1808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:48.583820 1808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: Sending DeleteTablet for 3 replicas of tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63
I20260807 15:14:48.584594 2414 tablet_service.cc:1514] Processing DeleteTablet for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:48 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:48814
I20260807 15:14:48.584710 2009 tablet_service.cc:1514] Processing DeleteTablet for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:48 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:35692
I20260807 15:14:48.585419 1808 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:14:48.585464 2214 tablet_service.cc:1514] Processing DeleteTablet for tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:48 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:36034
W20260807 15:14:48.585848 1808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:48.586045 2577 tablet_replica.cc:330] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: stopping tablet replica
I20260807 15:14:48.586337 2578 tablet_replica.cc:330] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: stopping tablet replica
I20260807 15:14:48.586398 2577 raft_consensus.cc:2229] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:48.586726 2577 raft_consensus.cc:2258] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:48.586756 2578 raft_consensus.cc:2229] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:48.587147 2578 raft_consensus.cc:2258] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:48.587558 2576 tablet_replica.cc:330] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: stopping tablet replica
I20260807 15:14:48.587826 2576 raft_consensus.cc:2229] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:48.588248 2576 raft_consensus.cc:2258] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:48.588850 2577 ts_tablet_manager.cc:1905] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:48.589768 2578 ts_tablet_manager.cc:1905] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:48.591059 2414 tablet_service.cc:1467] Processing CreateTablet for tablet 2040d0d18c2f4d668950611dcaced422 (DEFAULT_TABLE table=test-table [id=c345d9549e8242999c26b66749aae698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:48.591293 2009 tablet_service.cc:1467] Processing CreateTablet for tablet 2040d0d18c2f4d668950611dcaced422 (DEFAULT_TABLE table=test-table [id=c345d9549e8242999c26b66749aae698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:48.591354 2414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2040d0d18c2f4d668950611dcaced422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.591542 2009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2040d0d18c2f4d668950611dcaced422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.591738 2577 ts_tablet_manager.cc:1918] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:48.591823 2577 log.cc:1196] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/7af4fdd9d8ea4be8a64e3b15fb60aa63
I20260807 15:14:48.592144 2577 ts_tablet_manager.cc:1939] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P d67bfaa572ee40eab99fc3a23d368f62: Deleting consensus metadata
I20260807 15:14:48.592619 2578 ts_tablet_manager.cc:1918] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:48.592716 2578 log.cc:1196] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/7af4fdd9d8ea4be8a64e3b15fb60aa63
I20260807 15:14:48.593067 2578 ts_tablet_manager.cc:1939] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 55799da85e05420eb8c6b82305232453: Deleting consensus metadata
I20260807 15:14:48.593400 1790 catalog_manager.cc:4841] TS d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239): tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 (table test-table [id=43f74a505f144bc69e8194e3126f9c07]) successfully deleted
I20260807 15:14:48.593781 2544 tablet_bootstrap.cc:492] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Bootstrap starting.
I20260807 15:14:48.593896 2214 tablet_service.cc:1467] Processing CreateTablet for tablet 2040d0d18c2f4d668950611dcaced422 (DEFAULT_TABLE table=test-table [id=c345d9549e8242999c26b66749aae698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:48.594154 2214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2040d0d18c2f4d668950611dcaced422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.594321 2546 tablet_bootstrap.cc:492] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Bootstrap starting.
I20260807 15:14:48.594789 2544 tablet_bootstrap.cc:654] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.595328 2546 tablet_bootstrap.cc:654] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.595971 1790 catalog_manager.cc:4841] TS 55799da85e05420eb8c6b82305232453 (127.31.230.194:41729): tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 (table test-table [id=43f74a505f144bc69e8194e3126f9c07]) successfully deleted
I20260807 15:14:48.596249 2543 tablet_bootstrap.cc:492] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Bootstrap starting.
I20260807 15:14:48.597141 2546 tablet_bootstrap.cc:492] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: No bootstrap required, opened a new log
I20260807 15:14:48.597157 2543 tablet_bootstrap.cc:654] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.597270 2546 ts_tablet_manager.cc:1397] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:14:48.597783 2546 raft_consensus.cc:348] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.597918 2546 raft_consensus.cc:374] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.597950 2546 raft_consensus.cc:729] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d7e1eff31c41a3a5ae57f12b30335b, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.598086 2546 consensus_queue.cc:260] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.598819 2546 ts_tablet_manager.cc:1428] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:14:48.598922 2543 tablet_bootstrap.cc:492] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: No bootstrap required, opened a new log
I20260807 15:14:48.599000 2543 ts_tablet_manager.cc:1397] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:48.599539 2543 raft_consensus.cc:348] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.599653 2543 raft_consensus.cc:374] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.599689 2543 raft_consensus.cc:729] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55799da85e05420eb8c6b82305232453, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.599798 2544 tablet_bootstrap.cc:492] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: No bootstrap required, opened a new log
I20260807 15:14:48.599810 2543 consensus_queue.cc:260] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.599877 2544 ts_tablet_manager.cc:1397] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 15:14:48.600056 2543 ts_tablet_manager.cc:1428] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:48.600785 2544 raft_consensus.cc:348] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.600901 2544 raft_consensus.cc:374] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.600936 2544 raft_consensus.cc:729] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67bfaa572ee40eab99fc3a23d368f62, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.601049 2544 consensus_queue.cc:260] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.601332 2544 ts_tablet_manager.cc:1428] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:14:48.601579 2576 ts_tablet_manager.cc:1905] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:14:48.602932 2308 tablet.cc:2369] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:48.605161 2576 ts_tablet_manager.cc:1918] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:48.605288 2576 log.cc:1196] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/7af4fdd9d8ea4be8a64e3b15fb60aa63
I20260807 15:14:48.605634 2576 ts_tablet_manager.cc:1939] T 7af4fdd9d8ea4be8a64e3b15fb60aa63 P 31d7e1eff31c41a3a5ae57f12b30335b: Deleting consensus metadata
I20260807 15:14:48.606889 1791 catalog_manager.cc:4841] TS 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785): tablet 7af4fdd9d8ea4be8a64e3b15fb60aa63 (table test-table [id=43f74a505f144bc69e8194e3126f9c07]) successfully deleted
W20260807 15:14:48.610152 2110 tablet.cc:2369] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:48.620904 2555 raft_consensus.cc:482] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:48.621071 2555 raft_consensus.cc:504] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.622388 2555 leader_election.cc:288] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239), 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785)
I20260807 15:14:48.627635 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.623045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:51649 (local address 127.31.230.195:36785)
0807 15:14:48.623226 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.623232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.623486 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:14:48.623682 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.623685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.623734 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.623808 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.624161 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.624650 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.626334 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.626974 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.627111 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.627179 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.627425 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.627504 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:14:48.627658 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.622691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36785 (local address 127.31.230.194:51649)
0807 15:14:48.623385 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.623398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.623560 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.623836 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.623839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.623854 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.623981 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.623990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.624767 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.624773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.626195 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.626208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.627153 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.627156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.627264 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.627288 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.627405 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.627491 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":45,"thread_start_us":75,"threads_started":1}
I20260807 15:14:48.628497 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.622934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:48229 (local address 127.31.230.193:36239)
0807 15:14:48.623063 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.623068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.623304 (+ 236us) server_negotiation.cc:407] Connection header received
0807 15:14:48.623523 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.623528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.623593 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.623666 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:48.624008 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.624814 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.627244 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.627899 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.628021 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.628080 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.628288 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.628373 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:14:48.628640 2439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2040d0d18c2f4d668950611dcaced422" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" is_pre_election: true
I20260807 15:14:48.628924 2439 raft_consensus.cc:2454] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 0.
I20260807 15:14:48.629292 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.622619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36239 (local address 127.31.230.194:48229)
0807 15:14:48.623221 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.623236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.623385 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.623686 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.623689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.623703 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:48.623836 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.623843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.624928 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.624931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.626195 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.626208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.628030 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.628035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.628142 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.628161 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.628268 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.628355 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":36,"thread_start_us":100,"threads_started":1}
I20260807 15:14:48.629374 2148 leader_election.cc:302] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d7e1eff31c41a3a5ae57f12b30335b, 55799da85e05420eb8c6b82305232453; no voters:
I20260807 15:14:48.629717 2555 raft_consensus.cc:2781] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:48.629770 2555 raft_consensus.cc:482] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:48.629806 2555 raft_consensus.cc:3037] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.629770 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2040d0d18c2f4d668950611dcaced422" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67bfaa572ee40eab99fc3a23d368f62" is_pre_election: true
I20260807 15:14:48.629909 2036 raft_consensus.cc:2454] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 0.
I20260807 15:14:48.631121 2555 raft_consensus.cc:504] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.631501 2555 leader_election.cc:288] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 election: Requested vote from peers d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239), 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785)
I20260807 15:14:48.631878 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2040d0d18c2f4d668950611dcaced422" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67bfaa572ee40eab99fc3a23d368f62"
I20260807 15:14:48.631916 2439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2040d0d18c2f4d668950611dcaced422" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d7e1eff31c41a3a5ae57f12b30335b"
I20260807 15:14:48.631996 2036 raft_consensus.cc:3037] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.632072 2439 raft_consensus.cc:3037] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.633086 2439 raft_consensus.cc:2454] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 1.
I20260807 15:14:48.633172 2036 raft_consensus.cc:2454] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 1.
I20260807 15:14:48.633466 2147 leader_election.cc:302] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55799da85e05420eb8c6b82305232453, d67bfaa572ee40eab99fc3a23d368f62; no voters:
I20260807 15:14:48.633698 2555 raft_consensus.cc:2781] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:48.633971 2555 raft_consensus.cc:686] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 1 LEADER]: Becoming Leader. State: Replica: 55799da85e05420eb8c6b82305232453, State: Running, Role: LEADER
I20260807 15:14:48.634254 2555 consensus_queue.cc:237] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.636669 1809 catalog_manager.cc:5495] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55799da85e05420eb8c6b82305232453 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:48.681945 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.671603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43165 (local address 127.0.0.1:55890)
0807 15:14:48.671947 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.671964 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.672100 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.672378 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.672381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.672972 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:48.673120 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.673130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.674039 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.674043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.674940 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.674950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.675626 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.675630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.675740 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.676396 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:48.676411 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:48.678170 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:48.680238 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.680251 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.680257 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.680499 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.680788 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:48.680793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:48.680795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:48.680883 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:48.681279 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:48.681287 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:48.681405 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.681524 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.681728 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":142,"threads_started":1}
I20260807 15:14:48.681944 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.671666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.31.230.254:43165)
0807 15:14:48.671826 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.671832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.672048 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:14:48.672211 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.672213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.672261 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.672329 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.673243 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.673871 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.675157 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.675487 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.675586 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.678325 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.678329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.678331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.678352 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.680125 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.680606 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.680610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.680612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.680695 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.680974 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.680979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.680981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.681170 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.681279 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.681604 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.681818 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":64}
I20260807 15:14:48.686236 1809 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:55890:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:14:48.687024 1809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:48.692265 2009 tablet_service.cc:1467] Processing CreateTablet for tablet d19afc998290470e803f10b5267e2970 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b1a3b1cbb2f4896b0c309e3b5923e2b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:14:48.692490 2414 tablet_service.cc:1467] Processing CreateTablet for tablet d19afc998290470e803f10b5267e2970 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b1a3b1cbb2f4896b0c309e3b5923e2b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:14:48.692783 2009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d19afc998290470e803f10b5267e2970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.692910 2414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d19afc998290470e803f10b5267e2970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.693933 2214 tablet_service.cc:1467] Processing CreateTablet for tablet d19afc998290470e803f10b5267e2970 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b1a3b1cbb2f4896b0c309e3b5923e2b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:14:48.694314 2214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d19afc998290470e803f10b5267e2970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:48.695518 2546 tablet_bootstrap.cc:492] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: Bootstrap starting.
I20260807 15:14:48.696542 2546 tablet_bootstrap.cc:654] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.697829 2546 tablet_bootstrap.cc:492] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: No bootstrap required, opened a new log
I20260807 15:14:48.697930 2546 ts_tablet_manager.cc:1397] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:48.698402 2546 raft_consensus.cc:348] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.698514 2546 raft_consensus.cc:374] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.698547 2546 raft_consensus.cc:729] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d7e1eff31c41a3a5ae57f12b30335b, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.698832 2543 tablet_bootstrap.cc:492] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: Bootstrap starting.
I20260807 15:14:48.698854 2546 consensus_queue.cc:260] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.699304 2559 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.699461 2559 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.699836 2546 ts_tablet_manager.cc:1428] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:48.700007 2543 tablet_bootstrap.cc:654] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.701027 2544 tablet_bootstrap.cc:492] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: Bootstrap starting.
I20260807 15:14:48.702203 2544 tablet_bootstrap.cc:654] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:48.703629 2543 tablet_bootstrap.cc:492] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: No bootstrap required, opened a new log
I20260807 15:14:48.703636 2544 tablet_bootstrap.cc:492] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: No bootstrap required, opened a new log
I20260807 15:14:48.703719 2543 ts_tablet_manager.cc:1397] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 15:14:48.703719 2544 ts_tablet_manager.cc:1397] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:14:48.704180 2544 raft_consensus.cc:348] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.704180 2543 raft_consensus.cc:348] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.704293 2544 raft_consensus.cc:374] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.704293 2543 raft_consensus.cc:374] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:48.704329 2543 raft_consensus.cc:729] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55799da85e05420eb8c6b82305232453, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.704327 2544 raft_consensus.cc:729] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67bfaa572ee40eab99fc3a23d368f62, State: Initialized, Role: FOLLOWER
I20260807 15:14:48.704452 2543 consensus_queue.cc:260] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.704452 2544 consensus_queue.cc:260] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.704744 2544 ts_tablet_manager.cc:1428] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:48.704731 2543 ts_tablet_manager.cc:1428] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:48.704728 2555 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.704916 2555 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.704972 2556 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.705113 2556 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.719246 2555 raft_consensus.cc:482] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:48.719408 2555 raft_consensus.cc:504] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.719889 2555 leader_election.cc:288] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239), 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785)
I20260807 15:14:48.720319 2439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19afc998290470e803f10b5267e2970" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" is_pre_election: true
I20260807 15:14:48.720336 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19afc998290470e803f10b5267e2970" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67bfaa572ee40eab99fc3a23d368f62" is_pre_election: true
I20260807 15:14:48.720481 2439 raft_consensus.cc:2454] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 0.
I20260807 15:14:48.720498 2036 raft_consensus.cc:2454] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 0.
I20260807 15:14:48.720813 2147 leader_election.cc:302] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55799da85e05420eb8c6b82305232453, d67bfaa572ee40eab99fc3a23d368f62; no voters:
I20260807 15:14:48.721005 2555 raft_consensus.cc:2781] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:48.721081 2555 raft_consensus.cc:482] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:48.721133 2555 raft_consensus.cc:3037] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.722599 2555 raft_consensus.cc:504] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.723003 2555 leader_election.cc:288] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 election: Requested vote from peers d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239), 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785)
I20260807 15:14:48.723457 2036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19afc998290470e803f10b5267e2970" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67bfaa572ee40eab99fc3a23d368f62"
I20260807 15:14:48.723465 2439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19afc998290470e803f10b5267e2970" candidate_uuid: "55799da85e05420eb8c6b82305232453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d7e1eff31c41a3a5ae57f12b30335b"
I20260807 15:14:48.723594 2036 raft_consensus.cc:3037] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.723592 2439 raft_consensus.cc:3037] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:48.724598 2036 raft_consensus.cc:2454] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 1.
I20260807 15:14:48.724910 2147 leader_election.cc:302] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55799da85e05420eb8c6b82305232453, d67bfaa572ee40eab99fc3a23d368f62; no voters:
I20260807 15:14:48.724952 2439 raft_consensus.cc:2454] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55799da85e05420eb8c6b82305232453 in term 1.
I20260807 15:14:48.725244 2555 raft_consensus.cc:2781] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:48.725376 2555 raft_consensus.cc:686] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [term 1 LEADER]: Becoming Leader. State: Replica: 55799da85e05420eb8c6b82305232453, State: Running, Role: LEADER
I20260807 15:14:48.725564 2555 consensus_queue.cc:237] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } }
I20260807 15:14:48.726897 2588 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: TxnStatusTablet state changed. Reason: New leader 55799da85e05420eb8c6b82305232453. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.727041 2588 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:14:48.727867 2610 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:14:48.728080 2610 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:14:48.728493 1807 catalog_manager.cc:5495] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55799da85e05420eb8c6b82305232453 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:48.750953 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.745848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41729 (local address 127.0.0.1:36046)
0807 15:14:48.746003 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.746024 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.746139 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.746533 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.746536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.746545 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:14:48.746666 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.746671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.747820 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.747824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.748807 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.748818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.749609 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.749615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.749714 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.749772 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:14:48.750617 (+ 845us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:14:48.750635 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.750715 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.750813 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:14:48.750953 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.745956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.31.230.194:41729)
0807 15:14:48.746092 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.746097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.746111 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:14:48.746315 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.746319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.746390 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.746497 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:48.746802 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.747688 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.748949 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.749444 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.749588 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.749944 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:48.750502 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:48.750618 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.750746 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.750807 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:14:48.754560 2036 raft_consensus.cc:1261] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Refusing update from remote peer 55799da85e05420eb8c6b82305232453: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:48.754823 2439 raft_consensus.cc:1261] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Refusing update from remote peer 55799da85e05420eb8c6b82305232453: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:48.755095 2588 consensus_queue.cc:1035] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:48.755393 2555 consensus_queue.cc:1035] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:48.756825 2556 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: TxnStatusTablet state changed. Reason: New leader 55799da85e05420eb8c6b82305232453. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.756951 2556 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.758351 2559 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: TxnStatusTablet state changed. Reason: New leader 55799da85e05420eb8c6b82305232453. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.758477 2559 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.760370 2555 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.760478 2555 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:14:48.760584 2566 mvcc.cc:205] Tried to move back new op lower bound from 7315929861133094912 to 7315929861021847552. Current Snapshot: MvccSnapshot[applied={T|T < 7315929861133094912}]
I20260807 15:14:48.761699 2556 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.761817 2556 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P d67bfaa572ee40eab99fc3a23d368f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.765697 2588 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.765828 2588 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 55799da85e05420eb8c6b82305232453: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:14:48.767113 2559 tablet_replica.cc:439] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55799da85e05420eb8c6b82305232453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 } } peers { permanent_uuid: "55799da85e05420eb8c6b82305232453" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41729 } } peers { permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 } } }
I20260807 15:14:48.767247 2559 tablet_replica.cc:442] T d19afc998290470e803f10b5267e2970 P 31d7e1eff31c41a3a5ae57f12b30335b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:48.771636 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.767727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.31.230.194:41729)
0807 15:14:48.768080 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.768083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.768095 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:48.768127 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.768129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.768166 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.768259 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:48.768539 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.769334 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.770402 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.770745 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.770851 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.771152 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:48.771237 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:48.771314 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.771451 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.771507 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260807 15:14:48.782212 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.778806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:34147 (local address 127.31.230.194:41729)
0807 15:14:48.778922 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.778928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.779130 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:14:48.779293 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.779297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.779344 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.779416 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:48.779688 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.780168 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.781078 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.781295 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.781426 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.781793 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:48.781857 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:48.781913 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.782025 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.782065 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":39}
I20260807 15:14:48.782397 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.778741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41729 (local address 127.31.230.194:34147)
0807 15:14:48.779045 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:48.779057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:48.779178 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:48.779439 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:48.779441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:48.779448 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:14:48.779589 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.779594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.780332 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.780336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.780961 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:48.780969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.781464 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.781468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:48.781599 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.781654 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:14:48.781941 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:14:48.781947 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:14:48.782020 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:14:48.782105 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":21,"thread_start_us":124,"threads_started":1}
I20260807 15:14:48.789139 2439 raft_consensus.cc:1261] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Refusing update from remote peer 55799da85e05420eb8c6b82305232453: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:48.789266 2036 raft_consensus.cc:1261] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Refusing update from remote peer 55799da85e05420eb8c6b82305232453: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:48.789705 2624 consensus_queue.cc:1035] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67bfaa572ee40eab99fc3a23d368f62" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:48.789872 2555 consensus_queue.cc:1035] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d7e1eff31c41a3a5ae57f12b30335b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:48.835052 1809 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:48.835255 1809 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:48.837458 1809 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a35e276231234913b30fb55876aaf691: Sending DeleteTablet for 3 replicas of tablet 2040d0d18c2f4d668950611dcaced422
I20260807 15:14:48.838135 2009 tablet_service.cc:1514] Processing DeleteTablet for tablet 2040d0d18c2f4d668950611dcaced422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:48 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:35692
I20260807 15:14:48.838196 2414 tablet_service.cc:1514] Processing DeleteTablet for tablet 2040d0d18c2f4d668950611dcaced422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:48 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:48814
I20260807 15:14:48.838380 2576 tablet_replica.cc:330] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: stopping tablet replica
I20260807 15:14:48.838378 2577 tablet_replica.cc:330] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: stopping tablet replica
I20260807 15:14:48.838500 2576 raft_consensus.cc:2229] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:48.838493 2577 raft_consensus.cc:2229] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:48.838622 2576 raft_consensus.cc:2258] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:48.838644 2577 raft_consensus.cc:2258] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:48.839407 2577 ts_tablet_manager.cc:1905] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:48.840665 2576 ts_tablet_manager.cc:1905] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:48.842448 2577 ts_tablet_manager.cc:1918] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:48.842540 2577 log.cc:1196] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/2040d0d18c2f4d668950611dcaced422
I20260807 15:14:48.842818 2577 ts_tablet_manager.cc:1939] T 2040d0d18c2f4d668950611dcaced422 P d67bfaa572ee40eab99fc3a23d368f62: Deleting consensus metadata
I20260807 15:14:48.843484 2576 ts_tablet_manager.cc:1918] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:48.843569 2576 log.cc:1196] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/2040d0d18c2f4d668950611dcaced422
I20260807 15:14:48.843521 1790 catalog_manager.cc:4841] TS d67bfaa572ee40eab99fc3a23d368f62 (127.31.230.193:36239): tablet 2040d0d18c2f4d668950611dcaced422 (table test-table [id=c345d9549e8242999c26b66749aae698]) successfully deleted
I20260807 15:14:48.843824 2576 ts_tablet_manager.cc:1939] T 2040d0d18c2f4d668950611dcaced422 P 31d7e1eff31c41a3a5ae57f12b30335b: Deleting consensus metadata
I20260807 15:14:48.844542 1791 catalog_manager.cc:4841] TS 31d7e1eff31c41a3a5ae57f12b30335b (127.31.230.195:36785): tablet 2040d0d18c2f4d668950611dcaced422 (table test-table [id=c345d9549e8242999c26b66749aae698]) successfully deleted
I20260807 15:14:48.844671 2214 tablet_service.cc:1514] Processing DeleteTablet for tablet 2040d0d18c2f4d668950611dcaced422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:48 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:36034
I20260807 15:14:48.844882 2578 tablet_replica.cc:330] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: stopping tablet replica
I20260807 15:14:48.845027 2578 raft_consensus.cc:2229] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:48.845319 2578 raft_consensus.cc:2258] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:48.846056 2578 ts_tablet_manager.cc:1905] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:48.849085 2578 ts_tablet_manager.cc:1918] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:48.849179 2578 log.cc:1196] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/2040d0d18c2f4d668950611dcaced422
I20260807 15:14:48.849438 2578 ts_tablet_manager.cc:1939] T 2040d0d18c2f4d668950611dcaced422 P 55799da85e05420eb8c6b82305232453: Deleting consensus metadata
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[1749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:14:48.850190 1790 catalog_manager.cc:4841] TS 55799da85e05420eb8c6b82305232453 (127.31.230.194:41729): tablet 2040d0d18c2f4d668950611dcaced422 (table test-table [id=c345d9549e8242999c26b66749aae698]) successfully deleted
I20260807 15:14:48.853960 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.841681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.31.230.193:36239)
0807 15:14:48.841817 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.841822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.842073 (+ 251us) server_negotiation.cc:407] Connection header received
0807 15:14:48.842277 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.842283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.842347 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.842405 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.843886 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.844481 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.845767 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.846065 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.846151 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.850356 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.850367 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.850383 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.850410 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.852106 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.852551 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.852559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.852561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.852656 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.852947 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.852950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.852951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.853254 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.853463 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.853670 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.853793 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 15:14:48.854405 2098 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35704
I20260807 15:14:48.864944 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:48.856080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.31.230.254:43165)
0807 15:14:48.856210 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:48.856213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:48.856281 (+ 68us) server_negotiation.cc:407] Connection header received
0807 15:14:48.856412 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:48.856414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:48.856452 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:48.856516 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:48.857335 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.858136 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.859313 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:48.859676 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:48.859765 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:48.861390 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:48.861394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:48.861397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:48.861422 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:48.863151 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.863643 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.863646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.863648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.863734 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:48.864037 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:48.864040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:48.864042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:48.864253 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:48.864417 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:48.864630 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 15:14:48.864744 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 15:14:48.867599 1809 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55896
I20260807 15:14:48.869643 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 1912
I20260807 15:14:48.889693 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 2121
I20260807 15:14:48.901865 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 2319
I20260807 15:14:48.914243 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 1765
2026-08-07T15:14:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2962 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[2637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[2637](info): set up 2 sockets
Aug 07 15:14:48 dist-test-slave-gkw7 krb5kdc[2637](info): commencing operation
krb5kdc: starting...
W20260807 15:14:50.184502 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:14:50 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:14:50.312893 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39985
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:39985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:50.426976 2653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.427317 2653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.427399 2653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.431492 2653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:50.431589 2653 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:14:50.431620 2653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.431664 2653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.431703 2653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:50.431739 2653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:50.437216 2653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:39985
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:50.438464 2653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.439575 2653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.448421 2662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.448431 2660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.448439 2659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.448545 2653 server_base.cc:1034] running on GCE node
I20260807 15:14:50.449308 2653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.450502 2653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.451673 2653 hybrid_clock.cc:648] HybridClock initialized: now 1786115690451665 us; error 35 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.454583 2653 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:14:50.455542 2653 webserver.cc:460] Webserver started at http://127.31.230.254:33593/ using document root <none> and password file <none>
I20260807 15:14:50.456125 2653 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.456188 2653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.456418 2653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:50.458606 2653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "2c1d4d6e88ec43a6a9764aa5185ad286"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "91a6d31547722f060a54789afd18ad6f"
server_key_iv: "d87a3c434c437ef881dcea23446a10e0"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.459195 2653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "2c1d4d6e88ec43a6a9764aa5185ad286"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "91a6d31547722f060a54789afd18ad6f"
server_key_iv: "d87a3c434c437ef881dcea23446a10e0"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.463074 2653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:50.465520 2671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.466949 2653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:50.467095 2653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "2c1d4d6e88ec43a6a9764aa5185ad286"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "91a6d31547722f060a54789afd18ad6f"
server_key_iv: "d87a3c434c437ef881dcea23446a10e0"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.467213 2653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:50.480520 2653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:50.483598 2653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:50.483820 2653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:50.492487 2653 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:39985
I20260807 15:14:50.492517 2757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:39985 every 8 connection(s)
I20260807 15:14:50.493479 2653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:50.496326 2758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:50.500066 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 2653
I20260807 15:14:50.500192 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:14:50.500476 32667 external_mini_cluster.cc:1428] Setting key bb8cf93f6d58052c207e52b0d7328745
I20260807 15:14:50.502797 2758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Bootstrap starting.
I20260807 15:14:50.505064 2758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:50.505918 2758 log.cc:824] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:50.507941 2758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: No bootstrap required, opened a new log
Aug 07 15:14:50 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:50.511756 2758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39985 } }
I20260807 15:14:50.512032 2758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:50.512120 2758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1d4d6e88ec43a6a9764aa5185ad286, State: Initialized, Role: FOLLOWER
I20260807 15:14:50.512646 2758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39985 } }
I20260807 15:14:50.512809 2758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:50.512900 2758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:50.513019 2758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:50.514070 2758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39985 } }
I20260807 15:14:50.514449 2758 leader_election.cc:302] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c1d4d6e88ec43a6a9764aa5185ad286; no voters:
I20260807 15:14:50.514874 2758 leader_election.cc:288] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:50.514988 2766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:50.515211 2766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [term 1 LEADER]: Becoming Leader. State: Replica: 2c1d4d6e88ec43a6a9764aa5185ad286, State: Running, Role: LEADER
I20260807 15:14:50.515609 2766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39985 } }
I20260807 15:14:50.516013 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.502131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.31.230.254:39985)
0807 15:14:50.502596 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.502611 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.502649 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:14:50.503361 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.503383 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.503724 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.504078 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.504915 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.506111 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.507203 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.507693 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.507873 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.510524 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.510529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.510544 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.510576 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.512573 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.513101 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.513110 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.513118 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.513273 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.513629 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.513633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.513634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.514065 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.514304 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.514599 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.514952 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":145,"threads_started":1}
I20260807 15:14:50.516294 2758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:50.519168 2767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39985 } } }
I20260807 15:14:50.519124 2769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c1d4d6e88ec43a6a9764aa5185ad286. Latest consensus state: current_term: 1 leader_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1d4d6e88ec43a6a9764aa5185ad286" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39985 } } }
I20260807 15:14:50.519393 2769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:50.519393 2767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286 [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:50.522104 2792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:50.522192 2792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:50.522271 2793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:50.522943 2793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:50.528524 2793 catalog_manager.cc:1347] Generated new cluster ID: dbebdd29aecb45beb26343d82d6edcc9
I20260807 15:14:50.528609 2793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:50.545495 2793 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:50.546532 2793 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:50.553243 2793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Generated new TSK 0
I20260807 15:14:50.554033 2793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:14:50.594868 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39985
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:50.711508 2803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.711772 2803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.711820 2803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.715734 2803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.715790 2803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.715828 2803 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:14:50.715842 2803 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:14:50.715875 2803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:14:50.721508 2803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:39985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:50.722563 2803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.723569 2803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.733784 2809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.733822 2810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.733784 2812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.734112 2803 server_base.cc:1034] running on GCE node
I20260807 15:14:50.734658 2803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.735390 2803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.736601 2803 hybrid_clock.cc:648] HybridClock initialized: now 1786115690736557 us; error 50 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.739511 2803 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:14:50.740409 2803 webserver.cc:460] Webserver started at http://127.31.230.193:34697/ using document root <none> and password file <none>
I20260807 15:14:50.740993 2803 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.741086 2803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.741333 2803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:50.743480 2803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "59e753d95ef44a74b83f967816e4bd5c"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "4bd8a0f3fb7c7d3825eb05489578cf9d"
server_key_iv: "e0513e1a7ca9fa0d87eed6b2ee01cce8"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.744105 2803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "59e753d95ef44a74b83f967816e4bd5c"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "4bd8a0f3fb7c7d3825eb05489578cf9d"
server_key_iv: "e0513e1a7ca9fa0d87eed6b2ee01cce8"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.748034 2803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:14:50.750756 2819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.751979 2803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:50.752122 2803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "59e753d95ef44a74b83f967816e4bd5c"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "4bd8a0f3fb7c7d3825eb05489578cf9d"
server_key_iv: "e0513e1a7ca9fa0d87eed6b2ee01cce8"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.752241 2803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:50.767283 2803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:50.770856 2803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:50.771063 2803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:50.772729 2803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:50.772805 2803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.772866 2803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:50.772919 2803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:14:50 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:50.784503 2803 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:39265
I20260807 15:14:50.784618 2986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:39265 every 8 connection(s)
I20260807 15:14:50.785492 2803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:50.787395 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.773419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46157 (local address 127.31.230.254:39985)
0807 15:14:50.773612 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.773618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.775036 (+ 1418us) server_negotiation.cc:407] Connection header received
0807 15:14:50.776532 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.776535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.776615 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.776727 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.778200 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.778815 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.780180 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.780431 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.780551 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.783566 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.783570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.783572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.783598 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.785221 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.785848 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.785851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.785853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.785932 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.786349 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.786352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.786354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.786590 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.786684 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.787161 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.787270 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:14:50.788278 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.774351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.31.230.193:46157)
0807 15:14:50.774862 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:50.774923 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:50.776278 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:50.776918 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:50.776929 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:50.777480 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:50.778049 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.778062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.778945 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.778949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.780052 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.780062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.780602 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.780606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.780819 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.781572 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:50.781594 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:50.783386 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:50.785348 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.785354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.785369 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.785732 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.786067 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.786071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.786072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.786232 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.786704 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:50.786711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:50.786938 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 15:14:50.787096 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 15:14:50.787382 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":110,"threads_started":1}
I20260807 15:14:50.789624 2987 heartbeater.cc:347] Connected to a master server at 127.31.230.254:39985
I20260807 15:14:50.789768 2987 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:50.790328 2987 heartbeater.cc:511] Master 127.31.230.254:39985 requested a full tablet report, sending...
I20260807 15:14:50.791985 2692 ts_manager.cc:194] Registered new tserver with Master: 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265)
I20260807 15:14:50.792625 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 2803
I20260807 15:14:50.792763 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:50.793092 32667 external_mini_cluster.cc:1428] Setting key 61f28ad9d15657120fc12f62bf52e5b7
I20260807 15:14:50.793380 2692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:46157
I20260807 15:14:50.799791 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.791996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:48377 (local address 127.31.230.254:39985)
0807 15:14:50.792126 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.792130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.792260 (+ 130us) server_negotiation.cc:407] Connection header received
0807 15:14:50.792347 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.792350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.792389 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.792461 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.793018 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.793745 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.794927 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.795305 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.795433 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.796646 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.796649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.796651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.796674 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.798144 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.798638 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.798640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.798641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.798715 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.799013 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.799015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.799016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.799171 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.799270 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.799582 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.799674 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:14:50.799937 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.791890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.31.230.193:48377)
0807 15:14:50.792167 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:50.792179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:50.792269 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:50.792511 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:50.792514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:50.792747 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:50.792885 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.792895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.793897 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.793903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.794790 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.794801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.795441 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.795445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.795535 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.796084 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:50.796095 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:50.796483 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:50.798258 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.798265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.798270 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.798536 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.798807 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.798813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.798817 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.798922 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.799274 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:50.799280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:50.799394 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:14:50.799519 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 15:14:50.799692 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:14:50.824918 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39985
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:50.940701 2998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.941043 2998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.941125 2998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.945179 2998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.945283 2998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.945360 2998 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:14:50.945405 2998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:14:50.945475 2998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:14:50.951084 2998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:39985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:50.952246 2998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.953331 2998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.963582 3006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.963593 3014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.963683 2998 server_base.cc:1034] running on GCE node
W20260807 15:14:50.963593 3007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.964340 2998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.964968 2998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.966157 2998 hybrid_clock.cc:648] HybridClock initialized: now 1786115690966142 us; error 35 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.972445 2998 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:14:50.973464 2998 webserver.cc:460] Webserver started at http://127.31.230.194:41155/ using document root <none> and password file <none>
I20260807 15:14:50.974196 2998 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.974272 2998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.974478 2998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:50.976800 2998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "37a20202c6584fceff7ee1380c8cf444"
server_key_iv: "3ab75fd77cf82ea8d55c92452beea5c2"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.977468 2998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "37a20202c6584fceff7ee1380c8cf444"
server_key_iv: "3ab75fd77cf82ea8d55c92452beea5c2"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.982247 2998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:50.984989 3023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.986740 2998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:50.986904 2998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
format_stamp: "Formatted at 2026-08-07 15:14:50 on dist-test-slave-gkw7"
server_key: "37a20202c6584fceff7ee1380c8cf444"
server_key_iv: "3ab75fd77cf82ea8d55c92452beea5c2"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.987040 2998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:51.012555 2998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:51.016649 2998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:51.016876 2998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:51.018885 2998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:51.018967 2998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:51.019042 2998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:51.019107 2998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:14:51 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:51.031275 2998 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:42833
I20260807 15:14:51.031343 3193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:42833 every 8 connection(s)
I20260807 15:14:51.032582 2998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:51.032948 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 2998
I20260807 15:14:51.033078 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:51.033455 32667 external_mini_cluster.cc:1428] Setting key 1d882828ec7265e4d554cb1226a6de6e
I20260807 15:14:51.034135 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.019565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:43225 (local address 127.31.230.254:39985)
0807 15:14:51.019724 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.019731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.021168 (+ 1437us) server_negotiation.cc:407] Connection header received
0807 15:14:51.022415 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.022418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.022479 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.022565 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.023888 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.024462 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.025657 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.026117 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.026269 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.029618 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.029622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.029625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.029663 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.031501 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.032144 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.032146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.032148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.032235 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.032720 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.032723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.032725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.032936 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.033055 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.033766 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.033919 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:14:51.034907 3035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.020511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.31.230.194:43225)
0807 15:14:51.020987 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.021039 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.022194 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.022827 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.022833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.023161 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:51.023727 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.023740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.024597 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.024606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.025487 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.025496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.026315 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.026320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.026544 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.027342 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:51.027365 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:51.029413 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:51.031635 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.031642 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.031657 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.032021 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.032334 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.032337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.032340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.032537 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.033089 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:51.033096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:51.033446 (+ 350us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.033671 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.034019 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":136,"threads_started":1}
I20260807 15:14:51.036315 3194 heartbeater.cc:347] Connected to a master server at 127.31.230.254:39985
I20260807 15:14:51.036485 3194 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:51.037117 3194 heartbeater.cc:511] Master 127.31.230.254:39985 requested a full tablet report, sending...
I20260807 15:14:51.038317 2692 ts_manager.cc:194] Registered new tserver with Master: 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833)
I20260807 15:14:51.039052 2692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:43225
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:14:51.050153 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.038631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.31.230.194:47479)
0807 15:14:51.041802 (+ 3171us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.041820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.041938 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.042294 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.042300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.042560 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:51.042726 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.042735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.043661 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.043664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.044726 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.044740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.045282 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.045287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.045417 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.045966 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:51.045982 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:51.046449 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:51.048392 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.048397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.048399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.048608 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.048910 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.048915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.048916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.049014 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.049638 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:51.049641 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:51.049733 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.049844 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.049974 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3029,"thread_start_us":103,"threads_started":1}
I20260807 15:14:51.050344 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.038708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:47479 (local address 127.31.230.254:39985)
0807 15:14:51.038845 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.038850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.041915 (+ 3065us) server_negotiation.cc:407] Connection header received
0807 15:14:51.042038 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.042041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.042147 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.042226 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.042848 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.043522 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.044869 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.045145 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.045237 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.046596 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.046599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.046601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.046626 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.048258 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.048712 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.048716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.048718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.048811 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.049111 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.049115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.049117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.049421 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.049611 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.049938 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.050082 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:14:51.070132 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39985
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:51.195267 3206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:51.195593 3206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:51.195684 3206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:51.199991 3206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:51.200107 3206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:51.200192 3206 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:14:51.200240 3206 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:14:51.200306 3206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:14:51.205731 3206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:39985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:51.206889 3206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:51.208055 3206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:51.218104 3213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:51.218091 3215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:51.218091 3212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:51.218730 3206 server_base.cc:1034] running on GCE node
I20260807 15:14:51.219153 3206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:51.219801 3206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:51.220983 3206 hybrid_clock.cc:648] HybridClock initialized: now 1786115691220939 us; error 56 us; skew 500 ppm
Aug 07 15:14:51 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115691, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:51.224267 3206 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:14:51.225310 3206 webserver.cc:460] Webserver started at http://127.31.230.195:46157/ using document root <none> and password file <none>
I20260807 15:14:51.226002 3206 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:51.226116 3206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:51.226382 3206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:51.228757 3206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "3f3c95c4e2524a52b3084de72613f3ab"
format_stamp: "Formatted at 2026-08-07 15:14:51 on dist-test-slave-gkw7"
server_key: "053f105fb45306a22af46fe4d71249ff"
server_key_iv: "9ab8f0202ecd68daf8e888fc0cdcb3bf"
server_key_version: "encryptionkey@0"
I20260807 15:14:51.229420 3206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "3f3c95c4e2524a52b3084de72613f3ab"
format_stamp: "Formatted at 2026-08-07 15:14:51 on dist-test-slave-gkw7"
server_key: "053f105fb45306a22af46fe4d71249ff"
server_key_iv: "9ab8f0202ecd68daf8e888fc0cdcb3bf"
server_key_version: "encryptionkey@0"
I20260807 15:14:51.233851 3206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:51.236688 3227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:51.238224 3206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:51.238332 3206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "3f3c95c4e2524a52b3084de72613f3ab"
format_stamp: "Formatted at 2026-08-07 15:14:51 on dist-test-slave-gkw7"
server_key: "053f105fb45306a22af46fe4d71249ff"
server_key_iv: "9ab8f0202ecd68daf8e888fc0cdcb3bf"
server_key_version: "encryptionkey@0"
I20260807 15:14:51.238483 3206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:51.255813 3206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:51.259526 3206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:51.259737 3206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:51.261432 3206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:51.261485 3206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:51.261569 3206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:51.261598 3206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:14:51 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115691, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:51.273768 3206 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:37785
I20260807 15:14:51.275095 3206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:51.276155 3410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:37785 every 8 connection(s)
I20260807 15:14:51.278646 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 3206
I20260807 15:14:51.278764 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.262407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:36255 (local address 127.31.230.254:39985)
0807 15:14:51.262566 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.262570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.263710 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 15:14:51.264809 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.264815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.264893 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.264950 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.266443 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.267214 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.268620 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.268856 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.268990 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.273850 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.273857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.273861 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.273899 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.275940 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.276756 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.276759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.276761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.276861 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.277327 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.277331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.277333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.277709 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.277860 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.278460 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.278592 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:14:51.278795 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:51.279163 32667 external_mini_cluster.cc:1428] Setting key 2f153a759e792c8800de45cefd3863d5
I20260807 15:14:51.279795 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.262962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.31.230.195:36255)
0807 15:14:51.263515 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.263578 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.264495 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.265307 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.265314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.265765 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:51.266265 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.266279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.267364 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.267369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.268473 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.268484 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.269059 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.269063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.269301 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.269991 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:51.270009 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:51.273594 (+ 3585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:51.276135 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.276142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.276156 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.276633 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.277000 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.277003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.277005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.277191 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.277862 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:51.277868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:51.278190 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.278409 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.278747 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":135,"threads_started":1}
I20260807 15:14:51.281988 3411 heartbeater.cc:347] Connected to a master server at 127.31.230.254:39985
I20260807 15:14:51.282181 3411 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:51.282814 3411 heartbeater.cc:511] Master 127.31.230.254:39985 requested a full tablet report, sending...
I20260807 15:14:51.283949 2694 ts_manager.cc:194] Registered new tserver with Master: 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:51.284791 2694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:36255
I20260807 15:14:51.291874 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.284351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.31.230.195:37447)
0807 15:14:51.284673 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.284690 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.284808 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.285095 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.285099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.285300 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:51.285451 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.285461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.286258 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.286262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.287116 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.287127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.287574 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.287577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.287647 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.288164 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:51.288176 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:51.288511 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:51.290279 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.290282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.290283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.290491 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.290914 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.290921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.290926 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.291038 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.291409 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:51.291411 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:51.291500 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.291609 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.291734 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":109,"threads_started":1}
I20260807 15:14:51.291895 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.284390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:37447 (local address 127.31.230.254:39985)
0807 15:14:51.284510 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.284514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.284780 (+ 266us) server_negotiation.cc:407] Connection header received
0807 15:14:51.284917 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.284921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.284973 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.285039 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.285624 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.286119 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.287242 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.287452 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.287536 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.288683 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.288686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.288688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.288710 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.290170 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.290639 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.290642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.290643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.290730 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.291136 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.291138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.291139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.291310 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.291393 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.291677 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.291774 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:14:51.294284 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:14:51 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:14:51 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:51.322788 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.314152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.31.230.254:39985)
0807 15:14:51.314295 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.314299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.314314 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:14:51.314368 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.314371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.314435 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.314527 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.315239 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.315782 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.316662 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.316924 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.317025 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.319745 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.319748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.319750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.319777 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.321134 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.321618 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.321621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.321623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.321706 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.321971 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.321973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.321974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.322194 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.322303 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.322543 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.322656 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:14:51.325778 2694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:51.328797 2694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:51.343688 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.338136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42833 (local address 127.0.0.1:50520)
0807 15:14:51.338713 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.338726 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.338806 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.339618 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.339621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.339638 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.339880 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.339886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.340858 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.340860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.341981 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.341991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.343085 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.343090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.343219 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.343292 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.343403 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.343524 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":69,"spinlock_wait_cycles":768,"thread_start_us":118,"threads_started":1}
I20260807 15:14:51.343691 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.338605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.31.230.194:42833)
0807 15:14:51.339179 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.339190 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.339212 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:14:51.339288 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.339295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.339476 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.339620 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.340002 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.340760 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.342118 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.342957 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.343096 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.343229 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.343451 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.343545 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":130,"threads_started":1}
I20260807 15:14:51.344534 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.337685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39265 (local address 127.0.0.1:60894)
0807 15:14:51.338508 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.338562 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.338717 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.339546 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.339554 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.339585 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.339881 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.339888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.341927 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.341931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.343077 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.343086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.344135 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.344139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.344234 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.344256 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.344332 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.344403 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":113,"spinlock_wait_cycles":3328,"thread_start_us":98,"threads_started":1}
I20260807 15:14:51.344578 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.338073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.31.230.193:39265)
0807 15:14:51.338379 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.338387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.338874 (+ 487us) server_negotiation.cc:407] Connection header received
0807 15:14:51.338964 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.338972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.339199 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.339367 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.340887 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.341758 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.343216 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.344002 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.344153 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.344279 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.344369 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.344452 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":120,"threads_started":1}
I20260807 15:14:51.345935 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.337470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37785 (local address 127.0.0.1:36362)
0807 15:14:51.338532 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.338570 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.338716 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.339688 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.339691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.339704 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.339879 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.339886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.342933 (+ 3047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.342936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.344059 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.344072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.345539 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.345561 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.345651 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.345674 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.345749 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.345813 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":42,"spinlock_wait_cycles":7680,"thread_start_us":135,"threads_started":1}
I20260807 15:14:51.346359 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.337637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.31.230.195:37785)
0807 15:14:51.339063 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.339075 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.339097 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:14:51.339173 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.339178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.339385 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.339541 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.341590 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.342809 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.344213 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.345210 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.345366 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.345455 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.346102 (+ 647us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.346203 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1303,"thread_start_us":139,"threads_started":1}
I20260807 15:14:51.347292 2897 tablet_service.cc:1467] Processing CreateTablet for tablet 55065372e9f24a52a2c99be2dfec9d75 (DEFAULT_TABLE table=test-table [id=6b4e16298c16475892e413fd68cd3891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.347263 3088 tablet_service.cc:1467] Processing CreateTablet for tablet 55065372e9f24a52a2c99be2dfec9d75 (DEFAULT_TABLE table=test-table [id=6b4e16298c16475892e413fd68cd3891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.348196 2897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55065372e9f24a52a2c99be2dfec9d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.348294 3088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55065372e9f24a52a2c99be2dfec9d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.348881 3304 tablet_service.cc:1467] Processing CreateTablet for tablet 55065372e9f24a52a2c99be2dfec9d75 (DEFAULT_TABLE table=test-table [id=6b4e16298c16475892e413fd68cd3891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.349896 3304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55065372e9f24a52a2c99be2dfec9d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.355044 3454 tablet_bootstrap.cc:492] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Bootstrap starting.
I20260807 15:14:51.355894 3455 tablet_bootstrap.cc:492] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Bootstrap starting.
I20260807 15:14:51.357331 3456 tablet_bootstrap.cc:492] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Bootstrap starting.
I20260807 15:14:51.357623 3454 tablet_bootstrap.cc:654] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.358464 3455 tablet_bootstrap.cc:654] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.358556 3454 log.cc:824] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:51.359233 3455 log.cc:824] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:51.359323 3456 tablet_bootstrap.cc:654] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.360100 3456 log.cc:824] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:51.360762 3454 tablet_bootstrap.cc:492] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: No bootstrap required, opened a new log
I20260807 15:14:51.361017 3454 ts_tablet_manager.cc:1397] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 15:14:51.361213 3455 tablet_bootstrap.cc:492] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: No bootstrap required, opened a new log
I20260807 15:14:51.361426 3455 ts_tablet_manager.cc:1397] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:14:51.362072 3456 tablet_bootstrap.cc:492] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: No bootstrap required, opened a new log
I20260807 15:14:51.362284 3456 ts_tablet_manager.cc:1397] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260807 15:14:51.364369 3455 raft_consensus.cc:348] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.364609 3455 raft_consensus.cc:374] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.364662 3455 raft_consensus.cc:729] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b429fcc04d84cf89888e3731fe6b9c4, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.365118 3455 consensus_queue.cc:260] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.365842 3194 heartbeater.cc:503] Master 127.31.230.254:39985 was elected leader, sending a full tablet report...
I20260807 15:14:51.366159 3455 ts_tablet_manager.cc:1428] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:14:51.366178 3456 raft_consensus.cc:348] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.366451 3456 raft_consensus.cc:374] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.366499 3456 raft_consensus.cc:729] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.366896 3456 consensus_queue.cc:260] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.366995 3454 raft_consensus.cc:348] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.367252 3454 raft_consensus.cc:374] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.367306 3454 raft_consensus.cc:729] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.367693 3454 consensus_queue.cc:260] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.367969 3456 ts_tablet_manager.cc:1428] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 15:14:51.368307 3411 heartbeater.cc:503] Master 127.31.230.254:39985 was elected leader, sending a full tablet report...
I20260807 15:14:51.368366 2987 heartbeater.cc:503] Master 127.31.230.254:39985 was elected leader, sending a full tablet report...
I20260807 15:14:51.368572 3454 ts_tablet_manager.cc:1428] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260807 15:14:51.369395 3464 raft_consensus.cc:482] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:51.369555 3464 raft_consensus.cc:504] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.371536 3464 leader_election.cc:288] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265), 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833)
I20260807 15:14:51.374989 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.371005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:53689 (local address 127.31.230.193:39265)
0807 15:14:51.371114 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.371119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.371289 (+ 170us) server_negotiation.cc:407] Connection header received
0807 15:14:51.371416 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.371419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.371465 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.371533 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.371974 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.372714 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.374055 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.374471 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.374570 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.374610 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.374837 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.374894 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:14:51.375039 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.370948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39265 (local address 127.31.230.195:53689)
0807 15:14:51.371217 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.371233 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.371329 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.371662 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.371667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.371687 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.371830 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.371841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.372825 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.372828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.373926 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.373939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.374593 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.374596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.374702 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.374735 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.374812 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.374884 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260807 15:14:51.375905 2925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55065372e9f24a52a2c99be2dfec9d75" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e753d95ef44a74b83f967816e4bd5c" is_pre_election: true
I20260807 15:14:51.376212 2925 raft_consensus.cc:2454] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 0.
I20260807 15:14:51.376392 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.371983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:33181 (local address 127.31.230.194:42833)
0807 15:14:51.372109 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.372113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.372258 (+ 145us) server_negotiation.cc:407] Connection header received
0807 15:14:51.372450 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.372453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.372495 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.372542 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.372912 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.373389 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.375008 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.375652 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.375746 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.375789 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.376180 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.376275 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:14:51.376395 3468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.371791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42833 (local address 127.31.230.195:33181)
0807 15:14:51.372152 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.372170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.372327 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.372666 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.372668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.372682 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.372813 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.372819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.373501 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.373504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.374878 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.374887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.375887 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.375891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.376014 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.376039 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.376153 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.376241 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":106,"threads_started":1}
I20260807 15:14:51.376796 3230 leader_election.cc:302] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3c95c4e2524a52b3084de72613f3ab, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:51.377031 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55065372e9f24a52a2c99be2dfec9d75" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" is_pre_election: true
I20260807 15:14:51.377099 3467 raft_consensus.cc:2781] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:51.377206 3467 raft_consensus.cc:482] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:51.377314 3467 raft_consensus.cc:3037] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.377359 3122 raft_consensus.cc:2454] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 0.
I20260807 15:14:51.378546 3467 raft_consensus.cc:504] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.378898 3467 leader_election.cc:288] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 election: Requested vote from peers 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265), 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833)
I20260807 15:14:51.379245 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55065372e9f24a52a2c99be2dfec9d75" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
I20260807 15:14:51.379351 3122 raft_consensus.cc:3037] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.380384 3122 raft_consensus.cc:2454] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 1.
I20260807 15:14:51.380614 2925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55065372e9f24a52a2c99be2dfec9d75" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e753d95ef44a74b83f967816e4bd5c"
I20260807 15:14:51.380697 3232 leader_election.cc:302] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b429fcc04d84cf89888e3731fe6b9c4, 3f3c95c4e2524a52b3084de72613f3ab; no voters:
I20260807 15:14:51.380761 2925 raft_consensus.cc:3037] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.380892 3467 raft_consensus.cc:2781] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:51.381021 3467 raft_consensus.cc:686] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 LEADER]: Becoming Leader. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Running, Role: LEADER
I20260807 15:14:51.381395 3467 consensus_queue.cc:237] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } }
I20260807 15:14:51.382121 2925 raft_consensus.cc:2454] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 1.
I20260807 15:14:51.385027 2692 catalog_manager.cc:5495] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195). New cstate: current_term: 1 leader_uuid: "3f3c95c4e2524a52b3084de72613f3ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:14:51 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:51.427745 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.419385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.31.230.254:39985)
0807 15:14:51.419534 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.419540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.419753 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:14:51.419915 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.419917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.419963 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.420048 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.420702 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.421232 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.422101 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.422321 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.422429 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.424656 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.424660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.424663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.424692 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.426024 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.426482 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.426488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.426490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.426565 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.426843 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.426847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.426848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.427026 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.427141 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.427440 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.427595 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":49}
I20260807 15:14:51.427747 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.419337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39985 (local address 127.0.0.1:50988)
0807 15:14:51.419653 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.419665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.419817 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.420067 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.420069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.420493 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:51.420583 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.420589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.421351 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.421354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.421985 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.421994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.422432 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.422436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.422515 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.423052 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:51.423063 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:51.424523 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:51.426135 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.426146 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.426153 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.426382 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.426662 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.426665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.426667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.426747 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.427135 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:51.427141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:51.427263 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.427380 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.427540 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":123,"threads_started":1}
I20260807 15:14:51.431874 2692 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50988:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:14:51.432662 2692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:51.438272 3304 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8b6471b14648a88db4d359c555e51d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09e521ea755544efa29a9eb6796600d4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:14:51.438303 3088 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8b6471b14648a88db4d359c555e51d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09e521ea755544efa29a9eb6796600d4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:14:51.438136 2897 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8b6471b14648a88db4d359c555e51d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09e521ea755544efa29a9eb6796600d4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:14:51.438968 3088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8b6471b14648a88db4d359c555e51d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.438968 2897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8b6471b14648a88db4d359c555e51d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.438992 3304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8b6471b14648a88db4d359c555e51d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.441922 3456 tablet_bootstrap.cc:492] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: Bootstrap starting.
I20260807 15:14:51.442373 3455 tablet_bootstrap.cc:492] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: Bootstrap starting.
I20260807 15:14:51.442940 3456 tablet_bootstrap.cc:654] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.443583 3455 tablet_bootstrap.cc:654] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.443775 3454 tablet_bootstrap.cc:492] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: Bootstrap starting.
I20260807 15:14:51.444645 3456 tablet_bootstrap.cc:492] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: No bootstrap required, opened a new log
I20260807 15:14:51.444746 3456 ts_tablet_manager.cc:1397] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 15:14:51.445137 3454 tablet_bootstrap.cc:654] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.445288 3456 raft_consensus.cc:348] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.445401 3456 raft_consensus.cc:374] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.445436 3456 raft_consensus.cc:729] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.445442 3455 tablet_bootstrap.cc:492] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: No bootstrap required, opened a new log
I20260807 15:14:51.445519 3455 ts_tablet_manager.cc:1397] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:51.445603 3456 consensus_queue.cc:260] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.445915 3456 ts_tablet_manager.cc:1428] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:51.446004 3455 raft_consensus.cc:348] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.445978 3467 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.446110 3455 raft_consensus.cc:374] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.446142 3455 raft_consensus.cc:729] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b429fcc04d84cf89888e3731fe6b9c4, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.446157 3467 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.446260 3455 consensus_queue.cc:260] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.446552 3455 ts_tablet_manager.cc:1428] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:51.446580 3463 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.446775 3463 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.447852 3454 tablet_bootstrap.cc:492] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: No bootstrap required, opened a new log
I20260807 15:14:51.447973 3454 ts_tablet_manager.cc:1397] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:14:51.448547 3454 raft_consensus.cc:348] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.448724 3454 raft_consensus.cc:374] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.448770 3454 raft_consensus.cc:729] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.448931 3454 consensus_queue.cc:260] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.449277 3454 ts_tablet_manager.cc:1428] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:51.449225 3465 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.449373 3465 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.507488 3465 raft_consensus.cc:482] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:51.507622 3465 raft_consensus.cc:504] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.508777 3465 leader_election.cc:288] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833), 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:51.513015 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.509158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46783 (local address 127.31.230.195:37785)
0807 15:14:51.509314 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.509319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.509619 (+ 300us) server_negotiation.cc:407] Connection header received
0807 15:14:51.509762 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.509765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.509816 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.509908 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.510261 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.510776 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.511893 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.512393 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.512535 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.512599 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.512799 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.512886 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:14:51.513043 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.509062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37785 (local address 127.31.230.193:46783)
0807 15:14:51.509477 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.509489 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.509658 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.509920 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.509923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.509966 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.510133 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.510139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.510896 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.510901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.511764 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.511774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.512536 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.512540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.512652 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.512674 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.512783 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.512859 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":78,"spinlock_wait_cycles":8448,"thread_start_us":121,"threads_started":1}
I20260807 15:14:51.513170 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.509253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:35725 (local address 127.31.230.194:42833)
0807 15:14:51.509411 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.509415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.509570 (+ 155us) server_negotiation.cc:407] Connection header received
0807 15:14:51.509764 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.509767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.509820 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.509908 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.510600 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.511090 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.512076 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.512588 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.512712 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.512764 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.512962 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.513030 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 15:14:51.513458 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.509062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42833 (local address 127.31.230.193:35725)
0807 15:14:51.509452 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.509466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.509637 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.509919 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.509922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.509964 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.510472 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.510479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.511200 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.511206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.511950 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.511959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.512713 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.512716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.512829 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.512857 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.512943 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.513011 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":43,"spinlock_wait_cycles":31872,"thread_start_us":149,"threads_started":1}
I20260807 15:14:51.513774 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8b6471b14648a88db4d359c555e51d" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c95c4e2524a52b3084de72613f3ab" is_pre_election: true
I20260807 15:14:51.513959 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8b6471b14648a88db4d359c555e51d" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" is_pre_election: true
I20260807 15:14:51.514058 3337 raft_consensus.cc:2454] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 0.
I20260807 15:14:51.514123 3122 raft_consensus.cc:2454] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 0.
I20260807 15:14:51.514504 2824 leader_election.cc:302] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b429fcc04d84cf89888e3731fe6b9c4, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:51.514801 3465 raft_consensus.cc:2781] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:51.514864 3465 raft_consensus.cc:482] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:51.514911 3465 raft_consensus.cc:3037] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.515980 3465 raft_consensus.cc:504] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.516357 3465 leader_election.cc:288] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833), 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:51.516647 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8b6471b14648a88db4d359c555e51d" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
I20260807 15:14:51.516731 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8b6471b14648a88db4d359c555e51d" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c95c4e2524a52b3084de72613f3ab"
I20260807 15:14:51.516817 3122 raft_consensus.cc:3037] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.516832 3337 raft_consensus.cc:3037] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.517936 3337 raft_consensus.cc:2454] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 1.
I20260807 15:14:51.517980 3122 raft_consensus.cc:2454] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 1.
I20260807 15:14:51.518231 2824 leader_election.cc:302] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b429fcc04d84cf89888e3731fe6b9c4, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:51.518481 3465 raft_consensus.cc:2781] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:51.518776 3465 raft_consensus.cc:686] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Running, Role: LEADER
I20260807 15:14:51.519161 3465 consensus_queue.cc:237] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.521183 3486 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: TxnStatusTablet state changed. Reason: New leader 59e753d95ef44a74b83f967816e4bd5c. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.521298 3486 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:14:51.521634 3490 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:14:51.521860 3490 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:14:51.522464 2694 catalog_manager.cc:5495] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193). New cstate: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
W20260807 15:14:51.526898 3414 tablet.cc:2369] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:51.534543 3195 tablet.cc:2369] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:51.536908 2988 tablet.cc:2369] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:51.548324 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.544148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39265 (local address 127.0.0.1:60898)
0807 15:14:51.544288 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.544306 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.544415 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.544759 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.544761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.544767 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:14:51.544871 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.544876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.546016 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.546018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.546608 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.546616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.547201 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.547204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.547285 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.547332 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.548071 (+ 739us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.548083 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.548134 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.548206 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:14:51.548341 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.544247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.31.230.193:39265)
0807 15:14:51.544374 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.544379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.544395 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:14:51.544558 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.544562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.544627 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.544744 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.544973 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.545905 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.546723 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.547085 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.547184 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.547468 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.547978 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.548089 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.548148 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.548207 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:14:51.553647 3337 raft_consensus.cc:1261] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Refusing update from remote peer 59e753d95ef44a74b83f967816e4bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.553674 3122 raft_consensus.cc:1261] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Refusing update from remote peer 59e753d95ef44a74b83f967816e4bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.554332 3465 consensus_queue.cc:1035] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.554533 3486 consensus_queue.cc:1035] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.557837 3467 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: TxnStatusTablet state changed. Reason: New leader 59e753d95ef44a74b83f967816e4bd5c. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.557966 3467 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.559047 3463 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: TxnStatusTablet state changed. Reason: New leader 59e753d95ef44a74b83f967816e4bd5c. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.559165 3463 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.562933 3465 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.563053 3465 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:14:51.564317 3463 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.564427 3463 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 3b429fcc04d84cf89888e3731fe6b9c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.564931 3486 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.565047 3486 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 59e753d95ef44a74b83f967816e4bd5c: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:14:51.566527 3467 tablet_replica.cc:439] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } } }
I20260807 15:14:51.566660 3467 tablet_replica.cc:442] T 0f8b6471b14648a88db4d359c555e51d P 3f3c95c4e2524a52b3084de72613f3ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:14:51.567070 3499 mvcc.cc:205] Tried to move back new op lower bound from 7315929872596623360 to 7315929872465084416. Current Snapshot: MvccSnapshot[applied={T|T < 7315929872596623360}]
I20260807 15:14:51.575032 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.571383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.31.230.195:37785)
0807 15:14:51.571510 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.571513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.571525 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:51.571667 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.571670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.571761 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.571892 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.572165 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.572689 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.573588 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.573811 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.573929 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.574246 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.574638 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.574769 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.574837 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.574891 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:14:51.585378 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.582139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39265 (local address 127.31.230.195:51317)
0807 15:14:51.582299 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.582317 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.582405 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.582641 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.582644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.582651 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:14:51.582766 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.582779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.583521 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.583526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.584131 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.584139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.584633 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.584637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.584738 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.584780 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.585114 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.585121 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.585182 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.585265 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 15:14:51.585378 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.582275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:51317 (local address 127.31.230.193:39265)
0807 15:14:51.582400 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.582405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.582420 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:14:51.582496 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.582499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.582545 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.582633 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.582886 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.583407 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.584284 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.584522 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.584632 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.584947 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.585021 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.585093 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.585192 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.585242 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:14:51.592648 3122 raft_consensus.cc:1261] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Refusing update from remote peer 3f3c95c4e2524a52b3084de72613f3ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.592945 2925 raft_consensus.cc:1261] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Refusing update from remote peer 3f3c95c4e2524a52b3084de72613f3ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.593151 3467 consensus_queue.cc:1035] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.593513 3464 consensus_queue.cc:1035] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.597708 3496 mvcc.cc:205] Tried to move back new op lower bound from 7315929872758530048 to 7315929871901663232. Current Snapshot: MvccSnapshot[applied={T|T < 7315929872758530048}]
I20260807 15:14:51.598757 3497 mvcc.cc:205] Tried to move back new op lower bound from 7315929872758530048 to 7315929871901663232. Current Snapshot: MvccSnapshot[applied={T|T < 7315929872758530048}]
I20260807 15:14:51.617822 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.613566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37785 (local address 127.31.230.193:46849)
0807 15:14:51.613891 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.613903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.614031 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.614297 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.614299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.614322 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:14:51.614452 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.614457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.615311 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.615316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.616288 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.616301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.616891 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.616893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.616970 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.617017 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.617401 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.617410 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.617593 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.617679 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":104,"threads_started":1}
I20260807 15:14:51.617805 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.613621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46849 (local address 127.31.230.195:37785)
0807 15:14:51.613732 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.613737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.613984 (+ 247us) server_negotiation.cc:407] Connection header received
0807 15:14:51.614140 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.614143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.614197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.614286 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.614565 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.615187 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.616413 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.616771 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.616881 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.617166 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.617274 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.617347 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.617609 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.617666 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:14:51.642719 2694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:51.642953 2694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:51.646245 2694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Sending DeleteTablet for 3 replicas of tablet 55065372e9f24a52a2c99be2dfec9d75
I20260807 15:14:51.647115 3304 tablet_service.cc:1514] Processing DeleteTablet for tablet 55065372e9f24a52a2c99be2dfec9d75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:51 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:36362
I20260807 15:14:51.647331 3088 tablet_service.cc:1514] Processing DeleteTablet for tablet 55065372e9f24a52a2c99be2dfec9d75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:51 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50520
I20260807 15:14:51.647980 2897 tablet_service.cc:1514] Processing DeleteTablet for tablet 55065372e9f24a52a2c99be2dfec9d75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:51 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:60894
I20260807 15:14:51.648597 3525 tablet_replica.cc:330] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: stopping tablet replica
I20260807 15:14:51.648725 2694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:14:51.648840 3528 tablet_replica.cc:330] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: stopping tablet replica
W20260807 15:14:51.649204 2694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:51.649679 3528 raft_consensus.cc:2229] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:51.649874 3525 raft_consensus.cc:2229] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:51.650133 3528 raft_consensus.cc:2258] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:51.650600 3525 raft_consensus.cc:2258] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:51.652613 3525 ts_tablet_manager.cc:1905] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:51.654446 2897 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0b799a3350455d9090764628678112 (DEFAULT_TABLE table=test-table [id=c4049a2c21ef463fb9f7d8e39cb6832e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.654660 3304 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0b799a3350455d9090764628678112 (DEFAULT_TABLE table=test-table [id=c4049a2c21ef463fb9f7d8e39cb6832e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.654750 2897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0b799a3350455d9090764628678112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.654932 3304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0b799a3350455d9090764628678112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.657323 3454 tablet_bootstrap.cc:492] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Bootstrap starting.
I20260807 15:14:51.657612 3456 tablet_bootstrap.cc:492] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Bootstrap starting.
I20260807 15:14:51.658392 3454 tablet_bootstrap.cc:654] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.658601 3456 tablet_bootstrap.cc:654] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.660019 3088 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0b799a3350455d9090764628678112 (DEFAULT_TABLE table=test-table [id=c4049a2c21ef463fb9f7d8e39cb6832e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.660341 3088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0b799a3350455d9090764628678112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.660494 3525 ts_tablet_manager.cc:1918] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:51.660657 3525 log.cc:1196] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/55065372e9f24a52a2c99be2dfec9d75
I20260807 15:14:51.661084 3525 ts_tablet_manager.cc:1939] T 55065372e9f24a52a2c99be2dfec9d75 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting consensus metadata
I20260807 15:14:51.662487 2672 catalog_manager.cc:4841] TS 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785): tablet 55065372e9f24a52a2c99be2dfec9d75 (table test-table [id=6b4e16298c16475892e413fd68cd3891]) successfully deleted
I20260807 15:14:51.663141 3455 tablet_bootstrap.cc:492] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Bootstrap starting.
I20260807 15:14:51.663560 3454 tablet_bootstrap.cc:492] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: No bootstrap required, opened a new log
I20260807 15:14:51.663686 3454 ts_tablet_manager.cc:1397] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 15:14:51.663749 3456 tablet_bootstrap.cc:492] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: No bootstrap required, opened a new log
I20260807 15:14:51.663846 3456 ts_tablet_manager.cc:1397] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260807 15:14:51.664163 3455 tablet_bootstrap.cc:654] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.664183 3454 raft_consensus.cc:348] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.664354 3454 raft_consensus.cc:374] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.664397 3454 raft_consensus.cc:729] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.664397 3456 raft_consensus.cc:348] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.664536 3456 raft_consensus.cc:374] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.664597 3456 raft_consensus.cc:729] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.664546 3454 consensus_queue.cc:260] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.664752 3456 consensus_queue.cc:260] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.664863 3454 ts_tablet_manager.cc:1428] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:14:51.665067 3456 ts_tablet_manager.cc:1428] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260807 15:14:51.665660 3455 tablet_bootstrap.cc:492] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: No bootstrap required, opened a new log
I20260807 15:14:51.665750 3455 ts_tablet_manager.cc:1397] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:14:51.666214 3455 raft_consensus.cc:348] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.666328 3455 raft_consensus.cc:374] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.666363 3455 raft_consensus.cc:729] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b429fcc04d84cf89888e3731fe6b9c4, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.666487 3455 consensus_queue.cc:260] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.667418 3455 ts_tablet_manager.cc:1428] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:14:51.668080 3528 ts_tablet_manager.cc:1905] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:51.671394 3528 ts_tablet_manager.cc:1918] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:51.671502 3528 log.cc:1196] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/55065372e9f24a52a2c99be2dfec9d75
I20260807 15:14:51.671893 3528 ts_tablet_manager.cc:1939] T 55065372e9f24a52a2c99be2dfec9d75 P 59e753d95ef44a74b83f967816e4bd5c: Deleting consensus metadata
I20260807 15:14:51.673645 2673 catalog_manager.cc:4841] TS 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265): tablet 55065372e9f24a52a2c99be2dfec9d75 (table test-table [id=6b4e16298c16475892e413fd68cd3891]) successfully deleted
I20260807 15:14:51.674000 3526 tablet_replica.cc:330] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: stopping tablet replica
I20260807 15:14:51.674365 3526 raft_consensus.cc:2229] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:51.674613 3526 raft_consensus.cc:2258] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:51.676735 3526 ts_tablet_manager.cc:1905] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:51.680132 3526 ts_tablet_manager.cc:1918] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:51.680256 3526 log.cc:1196] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/55065372e9f24a52a2c99be2dfec9d75
I20260807 15:14:51.680613 3526 ts_tablet_manager.cc:1939] T 55065372e9f24a52a2c99be2dfec9d75 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting consensus metadata
I20260807 15:14:51.682291 2674 catalog_manager.cc:4841] TS 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833): tablet 55065372e9f24a52a2c99be2dfec9d75 (table test-table [id=6b4e16298c16475892e413fd68cd3891]) successfully deleted
I20260807 15:14:51.703617 3486 raft_consensus.cc:482] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:51.703778 3486 raft_consensus.cc:504] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.704198 3486 leader_election.cc:288] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833), 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:51.704742 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0b799a3350455d9090764628678112" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c95c4e2524a52b3084de72613f3ab" is_pre_election: true
I20260807 15:14:51.704907 3337 raft_consensus.cc:2454] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 0.
I20260807 15:14:51.705154 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0b799a3350455d9090764628678112" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" is_pre_election: true
I20260807 15:14:51.705278 2822 leader_election.cc:302] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3c95c4e2524a52b3084de72613f3ab, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:51.705425 3122 raft_consensus.cc:2454] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 0.
I20260807 15:14:51.705474 3486 raft_consensus.cc:2781] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:51.705585 3486 raft_consensus.cc:482] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:51.705651 3486 raft_consensus.cc:3037] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.706904 3486 raft_consensus.cc:504] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.707253 3486 leader_election.cc:288] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833), 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:51.707604 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0b799a3350455d9090764628678112" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
I20260807 15:14:51.707683 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0b799a3350455d9090764628678112" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c95c4e2524a52b3084de72613f3ab"
I20260807 15:14:51.707809 3337 raft_consensus.cc:3037] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.707825 3122 raft_consensus.cc:3037] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.709020 3337 raft_consensus.cc:2454] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 1.
I20260807 15:14:51.709041 3122 raft_consensus.cc:2454] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 1.
I20260807 15:14:51.709380 2822 leader_election.cc:302] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3c95c4e2524a52b3084de72613f3ab, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:51.709561 3486 raft_consensus.cc:2781] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:51.709678 3486 raft_consensus.cc:686] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Running, Role: LEADER
I20260807 15:14:51.709826 3486 consensus_queue.cc:237] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:51.711567 2689 catalog_manager.cc:5495] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193). New cstate: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:51.736227 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.732727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60914 (local address 127.31.230.193:39265)
0807 15:14:51.732843 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.732848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.732863 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:14:51.732987 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.732990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.733040 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.733131 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.733360 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.734198 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.735024 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.735381 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.735486 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.735695 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.735781 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.735852 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.736014 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.736081 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:14:51.745731 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.742320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39265 (local address 127.31.230.193:51345)
0807 15:14:51.742437 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.742451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.742528 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.742765 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.742767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.742773 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:14:51.742876 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.742880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.743861 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.743864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.744460 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.744468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.745011 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.745014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.745098 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.745123 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.745398 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.745403 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.745453 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.745570 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":33}
I20260807 15:14:51.745735 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.742392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:51345 (local address 127.31.230.193:39265)
0807 15:14:51.742497 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.742502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.742516 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:14:51.742619 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.742622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.742669 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.742748 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.742975 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.743760 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.744562 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.744893 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.744998 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.745235 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.745312 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.745383 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.745490 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.745568 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":42}
I20260807 15:14:51.751317 3122 raft_consensus.cc:1261] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Refusing update from remote peer 59e753d95ef44a74b83f967816e4bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.751503 3337 raft_consensus.cc:1261] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Refusing update from remote peer 59e753d95ef44a74b83f967816e4bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.751757 3465 consensus_queue.cc:1035] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.751951 3486 consensus_queue.cc:1035] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:14:51.767746 3496 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:14:53.339708 2689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:53.339869 2689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:53.342415 2689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Sending DeleteTablet for 3 replicas of tablet 6c0b799a3350455d9090764628678112
I20260807 15:14:53.343014 2897 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0b799a3350455d9090764628678112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:60894
I20260807 15:14:53.343076 3088 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0b799a3350455d9090764628678112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50520
I20260807 15:14:53.343123 3304 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0b799a3350455d9090764628678112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:36362
I20260807 15:14:53.343446 3567 tablet_replica.cc:330] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: stopping tablet replica
I20260807 15:14:53.343446 3564 tablet_replica.cc:330] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: stopping tablet replica
I20260807 15:14:53.343585 3567 raft_consensus.cc:2229] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:53.343699 3565 tablet_replica.cc:330] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: stopping tablet replica
I20260807 15:14:53.343735 3567 raft_consensus.cc:2258] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:53.343829 3565 raft_consensus.cc:2229] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:53.343946 3565 raft_consensus.cc:2258] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:53.344113 3564 raft_consensus.cc:2229] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:53.344429 3564 raft_consensus.cc:2258] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:53.345068 3564 ts_tablet_manager.cc:1905] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:53.347813 3565 ts_tablet_manager.cc:1905] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:53.347882 3567 ts_tablet_manager.cc:1905] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:53.348341 3564 ts_tablet_manager.cc:1918] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:14:53.348423 3564 log.cc:1196] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/6c0b799a3350455d9090764628678112
I20260807 15:14:53.348728 3564 ts_tablet_manager.cc:1939] T 6c0b799a3350455d9090764628678112 P 59e753d95ef44a74b83f967816e4bd5c: Deleting consensus metadata
I20260807 15:14:53.349503 2673 catalog_manager.cc:4841] TS 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265): tablet 6c0b799a3350455d9090764628678112 (table test-table [id=c4049a2c21ef463fb9f7d8e39cb6832e]) successfully deleted
I20260807 15:14:53.350755 3567 ts_tablet_manager.cc:1918] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:14:53.350855 3567 log.cc:1196] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/6c0b799a3350455d9090764628678112
I20260807 15:14:53.350939 3565 ts_tablet_manager.cc:1918] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:14:53.351025 3565 log.cc:1196] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/6c0b799a3350455d9090764628678112
I20260807 15:14:53.351186 3567 ts_tablet_manager.cc:1939] T 6c0b799a3350455d9090764628678112 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting consensus metadata
I20260807 15:14:53.351334 3565 ts_tablet_manager.cc:1939] T 6c0b799a3350455d9090764628678112 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting consensus metadata
I20260807 15:14:53.351898 2672 catalog_manager.cc:4841] TS 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785): tablet 6c0b799a3350455d9090764628678112 (table test-table [id=c4049a2c21ef463fb9f7d8e39cb6832e]) successfully deleted
I20260807 15:14:53.352047 2674 catalog_manager.cc:4841] TS 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833): tablet 6c0b799a3350455d9090764628678112 (table test-table [id=c4049a2c21ef463fb9f7d8e39cb6832e]) successfully deleted
Aug 07 15:14:53 dist-test-slave-gkw7 krb5kdc[2637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115693, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:14:53 dist-test-slave-gkw7 krb5kdc[2637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115693, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:53.374687 3576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:53.365344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.31.230.254:39985)
0807 15:14:53.365725 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:53.365743 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:53.365890 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:14:53.366033 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:53.366036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:53.366105 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:53.366209 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:53.367006 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:53.367558 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:53.368421 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:53.368645 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:53.368731 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:53.371261 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:53.371265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:53.371267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:53.371291 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:53.372803 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:53.373231 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:53.373234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:53.373236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:53.373319 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:53.373748 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:53.373750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:53.373751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:53.373933 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:53.374040 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:53.374345 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 15:14:53.374483 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":95,"threads_started":1}
I20260807 15:14:53.377511 2689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:53.378093 2689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:53.383001 2897 tablet_service.cc:1467] Processing CreateTablet for tablet 46d7e067f31d43d18c4ef770665b5cf3 (DEFAULT_TABLE table=test-table [id=23de7ec3831b436fba1b186a5aae3a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:53.383224 3304 tablet_service.cc:1467] Processing CreateTablet for tablet 46d7e067f31d43d18c4ef770665b5cf3 (DEFAULT_TABLE table=test-table [id=23de7ec3831b436fba1b186a5aae3a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:53.383353 2897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d7e067f31d43d18c4ef770665b5cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:53.383270 3088 tablet_service.cc:1467] Processing CreateTablet for tablet 46d7e067f31d43d18c4ef770665b5cf3 (DEFAULT_TABLE table=test-table [id=23de7ec3831b436fba1b186a5aae3a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:53.383517 3304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d7e067f31d43d18c4ef770665b5cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:53.383637 3088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d7e067f31d43d18c4ef770665b5cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:53.385680 3578 tablet_bootstrap.cc:492] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Bootstrap starting.
I20260807 15:14:53.386420 3581 tablet_bootstrap.cc:492] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Bootstrap starting.
I20260807 15:14:53.387251 3578 tablet_bootstrap.cc:654] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:53.387398 3581 tablet_bootstrap.cc:654] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:53.388038 3583 tablet_bootstrap.cc:492] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Bootstrap starting.
I20260807 15:14:53.388628 3578 tablet_bootstrap.cc:492] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: No bootstrap required, opened a new log
I20260807 15:14:53.388775 3578 ts_tablet_manager.cc:1397] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:53.388800 3581 tablet_bootstrap.cc:492] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: No bootstrap required, opened a new log
I20260807 15:14:53.388898 3581 ts_tablet_manager.cc:1397] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:53.388978 3583 tablet_bootstrap.cc:654] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:53.389348 3581 raft_consensus.cc:348] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.389468 3581 raft_consensus.cc:374] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:53.389509 3581 raft_consensus.cc:729] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Initialized, Role: FOLLOWER
I20260807 15:14:53.389463 3578 raft_consensus.cc:348] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.389650 3578 raft_consensus.cc:374] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:53.389681 3581 consensus_queue.cc:260] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.389741 3578 raft_consensus.cc:729] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b429fcc04d84cf89888e3731fe6b9c4, State: Initialized, Role: FOLLOWER
I20260807 15:14:53.389881 3578 consensus_queue.cc:260] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.389992 3581 ts_tablet_manager.cc:1428] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:53.390264 3583 tablet_bootstrap.cc:492] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: No bootstrap required, opened a new log
I20260807 15:14:53.390342 3583 ts_tablet_manager.cc:1397] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:53.390341 3578 ts_tablet_manager.cc:1428] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:53.390702 3583 raft_consensus.cc:348] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.390853 3583 raft_consensus.cc:374] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:53.390901 3583 raft_consensus.cc:729] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Initialized, Role: FOLLOWER
I20260807 15:14:53.391052 3583 consensus_queue.cc:260] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.391433 3583 ts_tablet_manager.cc:1428] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:53.514714 3588 raft_consensus.cc:482] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:53.514878 3588 raft_consensus.cc:504] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.515277 3588 leader_election.cc:288] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265), 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833)
I20260807 15:14:53.515758 2925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d7e067f31d43d18c4ef770665b5cf3" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e753d95ef44a74b83f967816e4bd5c" is_pre_election: true
I20260807 15:14:53.515787 3119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d7e067f31d43d18c4ef770665b5cf3" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" is_pre_election: true
I20260807 15:14:53.515939 2925 raft_consensus.cc:2454] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 0.
I20260807 15:14:53.516021 3119 raft_consensus.cc:2454] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 0.
I20260807 15:14:53.516333 3230 leader_election.cc:302] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3c95c4e2524a52b3084de72613f3ab, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:53.516530 3588 raft_consensus.cc:2781] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:53.516608 3588 raft_consensus.cc:482] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:53.516680 3588 raft_consensus.cc:3037] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:53.517939 3588 raft_consensus.cc:504] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.518342 3588 leader_election.cc:288] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 election: Requested vote from peers 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265), 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833)
I20260807 15:14:53.518750 2925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d7e067f31d43d18c4ef770665b5cf3" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e753d95ef44a74b83f967816e4bd5c"
I20260807 15:14:53.518803 3119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d7e067f31d43d18c4ef770665b5cf3" candidate_uuid: "3f3c95c4e2524a52b3084de72613f3ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
I20260807 15:14:53.518900 2925 raft_consensus.cc:3037] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:53.518931 3119 raft_consensus.cc:3037] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:53.519938 3119 raft_consensus.cc:2454] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 1.
I20260807 15:14:53.519948 2925 raft_consensus.cc:2454] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3c95c4e2524a52b3084de72613f3ab in term 1.
I20260807 15:14:53.520273 3232 leader_election.cc:302] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b429fcc04d84cf89888e3731fe6b9c4, 3f3c95c4e2524a52b3084de72613f3ab; no voters:
I20260807 15:14:53.520473 3588 raft_consensus.cc:2781] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:53.520778 3588 raft_consensus.cc:686] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 LEADER]: Becoming Leader. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Running, Role: LEADER
I20260807 15:14:53.520942 3588 consensus_queue.cc:237] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.522830 2692 catalog_manager.cc:5495] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195). New cstate: current_term: 1 leader_uuid: "3f3c95c4e2524a52b3084de72613f3ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } health_report { overall_health: HEALTHY } } }
I20260807 15:14:53.552894 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:53.549476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.31.230.195:37785)
0807 15:14:53.549830 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:53.549839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:53.549850 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:14:53.549890 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:53.549892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:53.549934 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:53.550031 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:53.550319 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:53.550893 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:53.551778 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:53.552014 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:53.552129 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:53.552411 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:53.552488 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:53.552569 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:53.552668 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:14:53.552733 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":164,"threads_started":1}
I20260807 15:14:53.559289 3119 raft_consensus.cc:1261] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Refusing update from remote peer 3f3c95c4e2524a52b3084de72613f3ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:53.559305 2925 raft_consensus.cc:1261] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Refusing update from remote peer 3f3c95c4e2524a52b3084de72613f3ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:53.559746 3588 consensus_queue.cc:1035] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:53.559948 3591 consensus_queue.cc:1035] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:53.593084 2689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:53.593245 2689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:53.595232 2689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Sending DeleteTablet for 3 replicas of tablet 46d7e067f31d43d18c4ef770665b5cf3
I20260807 15:14:53.595762 3088 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d7e067f31d43d18c4ef770665b5cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50520
I20260807 15:14:53.595754 2897 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d7e067f31d43d18c4ef770665b5cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:60894
I20260807 15:14:53.595817 3304 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d7e067f31d43d18c4ef770665b5cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:36362
I20260807 15:14:53.595947 3565 tablet_replica.cc:330] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: stopping tablet replica
I20260807 15:14:53.595990 3567 tablet_replica.cc:330] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: stopping tablet replica
I20260807 15:14:53.596077 3565 raft_consensus.cc:2229] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:53.596117 3567 raft_consensus.cc:2229] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:53.596200 3564 tablet_replica.cc:330] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: stopping tablet replica
I20260807 15:14:53.596221 3565 raft_consensus.cc:2258] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:53.596312 3564 raft_consensus.cc:2229] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:53.596334 3567 raft_consensus.cc:2258] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:53.596428 3564 raft_consensus.cc:2258] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:53.597112 3567 ts_tablet_manager.cc:1905] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:53.597126 3565 ts_tablet_manager.cc:1905] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:53.597065 2689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:53.597476 2689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:53.597836 3564 ts_tablet_manager.cc:1905] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:53.599511 3565 ts_tablet_manager.cc:1918] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:53.599567 3565 log.cc:1196] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/46d7e067f31d43d18c4ef770665b5cf3
I20260807 15:14:53.599759 3565 ts_tablet_manager.cc:1939] T 46d7e067f31d43d18c4ef770665b5cf3 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting consensus metadata
I20260807 15:14:53.599972 3567 ts_tablet_manager.cc:1918] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:53.600045 3567 log.cc:1196] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/46d7e067f31d43d18c4ef770665b5cf3
I20260807 15:14:53.600538 3567 ts_tablet_manager.cc:1939] T 46d7e067f31d43d18c4ef770665b5cf3 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting consensus metadata
I20260807 15:14:53.600657 3564 ts_tablet_manager.cc:1918] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:14:53.600728 3564 log.cc:1196] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/46d7e067f31d43d18c4ef770665b5cf3
I20260807 15:14:53.600915 2674 catalog_manager.cc:4841] TS 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833): tablet 46d7e067f31d43d18c4ef770665b5cf3 (table test-table [id=23de7ec3831b436fba1b186a5aae3a6b]) successfully deleted
I20260807 15:14:53.600976 3564 ts_tablet_manager.cc:1939] T 46d7e067f31d43d18c4ef770665b5cf3 P 59e753d95ef44a74b83f967816e4bd5c: Deleting consensus metadata
I20260807 15:14:53.601846 2673 catalog_manager.cc:4841] TS 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265): tablet 46d7e067f31d43d18c4ef770665b5cf3 (table test-table [id=23de7ec3831b436fba1b186a5aae3a6b]) successfully deleted
I20260807 15:14:53.601990 2672 catalog_manager.cc:4841] TS 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785): tablet 46d7e067f31d43d18c4ef770665b5cf3 (table test-table [id=23de7ec3831b436fba1b186a5aae3a6b]) successfully deleted
I20260807 15:14:53.603734 3088 tablet_service.cc:1467] Processing CreateTablet for tablet da5091e300274744a94589e1bb8b3565 (DEFAULT_TABLE table=test-table [id=726182b36fea4fff86cd9c66e60f7530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:53.603739 2897 tablet_service.cc:1467] Processing CreateTablet for tablet da5091e300274744a94589e1bb8b3565 (DEFAULT_TABLE table=test-table [id=726182b36fea4fff86cd9c66e60f7530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:53.603909 3304 tablet_service.cc:1467] Processing CreateTablet for tablet da5091e300274744a94589e1bb8b3565 (DEFAULT_TABLE table=test-table [id=726182b36fea4fff86cd9c66e60f7530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:53.604025 3088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da5091e300274744a94589e1bb8b3565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:53.604179 3304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da5091e300274744a94589e1bb8b3565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:53.604270 2897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da5091e300274744a94589e1bb8b3565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:53.606225 3583 tablet_bootstrap.cc:492] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Bootstrap starting.
I20260807 15:14:53.606422 3578 tablet_bootstrap.cc:492] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: Bootstrap starting.
I20260807 15:14:53.606468 3581 tablet_bootstrap.cc:492] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c: Bootstrap starting.
I20260807 15:14:53.607101 3583 tablet_bootstrap.cc:654] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:53.607213 3578 tablet_bootstrap.cc:654] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:53.607239 3581 tablet_bootstrap.cc:654] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:53.608413 3581 tablet_bootstrap.cc:492] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c: No bootstrap required, opened a new log
I20260807 15:14:53.608412 3583 tablet_bootstrap.cc:492] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: No bootstrap required, opened a new log
I20260807 15:14:53.608507 3583 ts_tablet_manager.cc:1397] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:53.608507 3581 ts_tablet_manager.cc:1397] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:14:53.608906 3581 raft_consensus.cc:348] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.608906 3583 raft_consensus.cc:348] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.609030 3581 raft_consensus.cc:374] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:53.609030 3583 raft_consensus.cc:374] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:53.609066 3583 raft_consensus.cc:729] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3c95c4e2524a52b3084de72613f3ab, State: Initialized, Role: FOLLOWER
I20260807 15:14:53.609066 3581 raft_consensus.cc:729] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Initialized, Role: FOLLOWER
I20260807 15:14:53.609200 3583 consensus_queue.cc:260] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.609200 3581 consensus_queue.cc:260] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.609476 3581 ts_tablet_manager.cc:1428] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:53.609476 3583 ts_tablet_manager.cc:1428] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:53.610342 3578 tablet_bootstrap.cc:492] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: No bootstrap required, opened a new log
I20260807 15:14:53.610430 3578 ts_tablet_manager.cc:1397] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 15:14:53.610852 3578 raft_consensus.cc:348] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.610970 3578 raft_consensus.cc:374] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:53.611011 3578 raft_consensus.cc:729] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b429fcc04d84cf89888e3731fe6b9c4, State: Initialized, Role: FOLLOWER
I20260807 15:14:53.611145 3578 consensus_queue.cc:260] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.611428 3578 ts_tablet_manager.cc:1428] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:53.730943 3545 raft_consensus.cc:482] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:53.731105 3545 raft_consensus.cc:504] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.731601 3545 leader_election.cc:288] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833), 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:53.732055 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5091e300274744a94589e1bb8b3565" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" is_pre_election: true
I20260807 15:14:53.732234 3122 raft_consensus.cc:2454] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 0.
I20260807 15:14:53.732174 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5091e300274744a94589e1bb8b3565" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c95c4e2524a52b3084de72613f3ab" is_pre_election: true
I20260807 15:14:53.732360 3337 raft_consensus.cc:2454] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 0.
I20260807 15:14:53.732530 2824 leader_election.cc:302] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b429fcc04d84cf89888e3731fe6b9c4, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:53.732775 3545 raft_consensus.cc:2781] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:53.732841 3545 raft_consensus.cc:482] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:53.732888 3545 raft_consensus.cc:3037] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:53.734158 3545 raft_consensus.cc:504] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.734489 3545 leader_election.cc:288] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 election: Requested vote from peers 3b429fcc04d84cf89888e3731fe6b9c4 (127.31.230.194:42833), 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785)
I20260807 15:14:53.735072 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5091e300274744a94589e1bb8b3565" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c95c4e2524a52b3084de72613f3ab"
I20260807 15:14:53.735219 3337 raft_consensus.cc:3037] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:53.735203 3122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5091e300274744a94589e1bb8b3565" candidate_uuid: "59e753d95ef44a74b83f967816e4bd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b429fcc04d84cf89888e3731fe6b9c4"
I20260807 15:14:53.735335 3122 raft_consensus.cc:3037] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:53.736442 3122 raft_consensus.cc:2454] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 1.
I20260807 15:14:53.736645 3337 raft_consensus.cc:2454] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e753d95ef44a74b83f967816e4bd5c in term 1.
I20260807 15:14:53.737084 2822 leader_election.cc:302] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3c95c4e2524a52b3084de72613f3ab, 59e753d95ef44a74b83f967816e4bd5c; no voters:
I20260807 15:14:53.738200 3545 raft_consensus.cc:2781] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:53.738299 3545 raft_consensus.cc:686] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [term 1 LEADER]: Becoming Leader. State: Replica: 59e753d95ef44a74b83f967816e4bd5c, State: Running, Role: LEADER
I20260807 15:14:53.738446 3545 consensus_queue.cc:237] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } }
I20260807 15:14:53.740453 2689 catalog_manager.cc:5495] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193). New cstate: current_term: 1 leader_uuid: "59e753d95ef44a74b83f967816e4bd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e753d95ef44a74b83f967816e4bd5c" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:53.773461 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:53.768878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.31.230.193:39265)
0807 15:14:53.769176 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:53.769185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:53.769200 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:14:53.769245 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:53.769249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:53.769300 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:53.769408 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:53.769768 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:53.770647 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:53.772270 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:53.772518 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:53.772606 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:53.772951 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:53.773026 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:53.773084 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:53.773231 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:14:53.773280 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
I20260807 15:14:53.779601 3337 raft_consensus.cc:1261] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Refusing update from remote peer 59e753d95ef44a74b83f967816e4bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:53.779716 3122 raft_consensus.cc:1261] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Refusing update from remote peer 59e753d95ef44a74b83f967816e4bd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:53.780017 3545 consensus_queue.cc:1035] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3c95c4e2524a52b3084de72613f3ab" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:53.780164 3539 consensus_queue.cc:1035] T da5091e300274744a94589e1bb8b3565 P 59e753d95ef44a74b83f967816e4bd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b429fcc04d84cf89888e3731fe6b9c4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:14:53.798190 3537 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:14:55.352706 2692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:55.352908 2692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:55.355854 2692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c1d4d6e88ec43a6a9764aa5185ad286: Sending DeleteTablet for 3 replicas of tablet da5091e300274744a94589e1bb8b3565
I20260807 15:14:55.356523 2897 tablet_service.cc:1514] Processing DeleteTablet for tablet da5091e300274744a94589e1bb8b3565 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:55 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:60894
I20260807 15:14:55.356545 3088 tablet_service.cc:1514] Processing DeleteTablet for tablet da5091e300274744a94589e1bb8b3565 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:55 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50520
I20260807 15:14:55.356618 3304 tablet_service.cc:1514] Processing DeleteTablet for tablet da5091e300274744a94589e1bb8b3565 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:55 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:36362
I20260807 15:14:55.356999 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 2803
I20260807 15:14:55.362362 3638 tablet_replica.cc:330] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: stopping tablet replica
I20260807 15:14:55.362550 3638 raft_consensus.cc:2229] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:55.362700 3638 raft_consensus.cc:2258] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:55.363438 3638 ts_tablet_manager.cc:1905] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:55.365844 3638 ts_tablet_manager.cc:1918] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:14:55.365942 3638 log.cc:1196] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/da5091e300274744a94589e1bb8b3565
I20260807 15:14:55.366298 3638 ts_tablet_manager.cc:1939] T da5091e300274744a94589e1bb8b3565 P 3f3c95c4e2524a52b3084de72613f3ab: Deleting consensus metadata
I20260807 15:14:55.367244 2672 catalog_manager.cc:4841] TS 3f3c95c4e2524a52b3084de72613f3ab (127.31.230.195:37785): tablet da5091e300274744a94589e1bb8b3565 (table test-table [id=726182b36fea4fff86cd9c66e60f7530]) successfully deleted
I20260807 15:14:55.370703 3636 tablet_replica.cc:330] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: stopping tablet replica
I20260807 15:14:55.370903 3636 raft_consensus.cc:2229] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:55.371059 3636 raft_consensus.cc:2258] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:55.372009 3636 ts_tablet_manager.cc:1905] T da5091e300274744a94589e1bb8b3565 P 3b429fcc04d84cf89888e3731fe6b9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:14:55.374011 2673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:39265 (error 108)
I20260807 15:14:55.374210 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 2998
I20260807 15:14:55.377489 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:55.376878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39265 (local address 127.0.0.1:60920)
0807 15:14:55.377241 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:55.377345 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":96,"threads_started":1}
W20260807 15:14:55.377729 2673 catalog_manager.cc:4584] TS 59e753d95ef44a74b83f967816e4bd5c (127.31.230.193:39265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da5091e300274744a94589e1bb8b3565: Network error: Client connection negotiation failed: client connection to 127.31.230.193:39265: connect: Connection refused (error 111)
I20260807 15:14:55.383739 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 3206
I20260807 15:14:55.384162 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:55.383778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42833 (local address 127.0.0.1:50530)
0807 15:14:55.384002 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:55.384064 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.194:42833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 15:14:55.402170 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 2653
2026-08-07T15:14:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6487 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:55 dist-test-slave-gkw7 krb5kdc[3648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:55 dist-test-slave-gkw7 krb5kdc[3648](info): set up 2 sockets
Aug 07 15:14:55 dist-test-slave-gkw7 krb5kdc[3648](info): commencing operation
krb5kdc: starting...
W20260807 15:14:56.670313 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:14:56 dist-test-slave-gkw7 krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:14:56.806598 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33181
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:33181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:56.923250 3667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:56.923517 3667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:56.923556 3667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:56.927441 3667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:56.927505 3667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:56.927524 3667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:56.927541 3667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:56.927556 3667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:56.932834 3667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:33181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:56.934046 3667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:56.935078 3667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:56.944594 3674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:56.944599 3673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:56.944602 3678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:56.944608 3667 server_base.cc:1034] running on GCE node
I20260807 15:14:56.945443 3667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:56.946688 3667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:56.947886 3667 hybrid_clock.cc:648] HybridClock initialized: now 1786115696947865 us; error 34 us; skew 500 ppm
Aug 07 15:14:56 dist-test-slave-gkw7 krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115696, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:56.954079 3667 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:14:56.955084 3667 webserver.cc:460] Webserver started at http://127.31.230.254:37075/ using document root <none> and password file <none>
I20260807 15:14:56.955683 3667 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:56.955755 3667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:56.955981 3667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:56.958196 3667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "b932fc4115924c639a91cca2e6d83b55"
format_stamp: "Formatted at 2026-08-07 15:14:56 on dist-test-slave-gkw7"
server_key: "b3633eb4fc7fb19faf8f421068e4e074"
server_key_iv: "5bb74916fd04f7cd882d03633d7765c7"
server_key_version: "encryptionkey@0"
I20260807 15:14:56.958801 3667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "b932fc4115924c639a91cca2e6d83b55"
format_stamp: "Formatted at 2026-08-07 15:14:56 on dist-test-slave-gkw7"
server_key: "b3633eb4fc7fb19faf8f421068e4e074"
server_key_iv: "5bb74916fd04f7cd882d03633d7765c7"
server_key_version: "encryptionkey@0"
I20260807 15:14:56.962795 3667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:56.965317 3688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:56.966881 3667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:56.967032 3667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "b932fc4115924c639a91cca2e6d83b55"
format_stamp: "Formatted at 2026-08-07 15:14:56 on dist-test-slave-gkw7"
server_key: "b3633eb4fc7fb19faf8f421068e4e074"
server_key_iv: "5bb74916fd04f7cd882d03633d7765c7"
server_key_version: "encryptionkey@0"
I20260807 15:14:56.967151 3667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:56.985471 3667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:56.988708 3667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:56.988940 3667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:56.996642 3667 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:33181
I20260807 15:14:56.996687 3763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:33181 every 8 connection(s)
I20260807 15:14:56.997710 3667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:57.000638 3764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:57.004696 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 3667
I20260807 15:14:57.004837 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:14:57.005115 32667 external_mini_cluster.cc:1428] Setting key 9949149ed6559bb585a5683a42ceca5e
I20260807 15:14:57.006654 3764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55: Bootstrap starting.
I20260807 15:14:57.009182 3764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:57.010043 3764 log.cc:824] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:57.012578 3764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55: No bootstrap required, opened a new log
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:57.017299 3764 raft_consensus.cc:348] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932fc4115924c639a91cca2e6d83b55" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33181 } }
I20260807 15:14:57.017635 3764 raft_consensus.cc:374] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:57.017719 3764 raft_consensus.cc:729] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b932fc4115924c639a91cca2e6d83b55, State: Initialized, Role: FOLLOWER
I20260807 15:14:57.018316 3764 consensus_queue.cc:260] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932fc4115924c639a91cca2e6d83b55" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33181 } }
I20260807 15:14:57.018512 3764 raft_consensus.cc:388] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:57.018603 3764 raft_consensus.cc:482] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:57.018726 3764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:57.020053 3764 raft_consensus.cc:504] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932fc4115924c639a91cca2e6d83b55" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33181 } }
I20260807 15:14:57.020423 3764 leader_election.cc:302] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b932fc4115924c639a91cca2e6d83b55; no voters:
I20260807 15:14:57.020787 3764 leader_election.cc:288] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:57.020903 3770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:57.020941 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.006648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.31.230.254:33181)
0807 15:14:57.007136 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:57.007152 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:57.007190 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:14:57.007835 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:57.007853 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:57.008182 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:57.008470 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:57.009445 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.010667 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.012020 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.012490 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.012608 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.015653 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:57.015659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:57.015673 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:57.015700 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:57.017681 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.018240 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.018245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.018250 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.018368 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.018719 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.018722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.018737 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.019195 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:57.019365 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:57.019682 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 15:14:57.019980 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":193,"threads_started":1}
I20260807 15:14:57.021117 3770 raft_consensus.cc:686] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [term 1 LEADER]: Becoming Leader. State: Replica: b932fc4115924c639a91cca2e6d83b55, State: Running, Role: LEADER
I20260807 15:14:57.021452 3770 consensus_queue.cc:237] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932fc4115924c639a91cca2e6d83b55" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33181 } }
I20260807 15:14:57.022269 3764 sys_catalog.cc:564] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:57.022806 3773 sys_catalog.cc:455] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b932fc4115924c639a91cca2e6d83b55. Latest consensus state: current_term: 1 leader_uuid: "b932fc4115924c639a91cca2e6d83b55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932fc4115924c639a91cca2e6d83b55" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33181 } } }
I20260807 15:14:57.023027 3773 sys_catalog.cc:458] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:57.023322 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b932fc4115924c639a91cca2e6d83b55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932fc4115924c639a91cca2e6d83b55" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33181 } } }
I20260807 15:14:57.023465 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:57.023802 3786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:57.026896 3786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:57.033028 3786 catalog_manager.cc:1347] Generated new cluster ID: 713dfc9271994b46b37e0b876219237b
I20260807 15:14:57.033113 3786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:57.045073 3786 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:57.046413 3786 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:57.058617 3786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b932fc4115924c639a91cca2e6d83b55: Generated new TSK 0
I20260807 15:14:57.059399 3786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:14:57.101665 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33181
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:57.228251 3806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:57.228705 3806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:57.228829 3806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:57.233403 3806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:57.233508 3806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:57.233635 3806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:14:57.239102 3806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:57.240321 3806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:57.241451 3806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:57.251627 3817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:57.251632 3815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:57.251752 3806 server_base.cc:1034] running on GCE node
W20260807 15:14:57.251641 3814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:57.252331 3806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:57.253006 3806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:57.254184 3806 hybrid_clock.cc:648] HybridClock initialized: now 1786115697254149 us; error 55 us; skew 500 ppm
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115697, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:57.257109 3806 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:14:57.258062 3806 webserver.cc:460] Webserver started at http://127.31.230.193:39987/ using document root <none> and password file <none>
I20260807 15:14:57.258672 3806 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:57.258771 3806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:57.259023 3806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:57.261149 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "43f73a84224b4a76b838638457f49633"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "568a0a66851f1ac55b425eb7b0310ffa"
server_key_iv: "6da69e16a11922dc036fca3f8aa2158a"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.261821 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "43f73a84224b4a76b838638457f49633"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "568a0a66851f1ac55b425eb7b0310ffa"
server_key_iv: "6da69e16a11922dc036fca3f8aa2158a"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.265719 3806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:57.268371 3829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.269773 3806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:57.269920 3806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "43f73a84224b4a76b838638457f49633"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "568a0a66851f1ac55b425eb7b0310ffa"
server_key_iv: "6da69e16a11922dc036fca3f8aa2158a"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.270040 3806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:57.318782 3806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:57.322433 3806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:57.322660 3806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:57.323382 3806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:57.324343 3806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:57.324417 3806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.324492 3806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:57.324568 3806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.333810 3806 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:43433
I20260807 15:14:57.333845 3985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:43433 every 8 connection(s)
I20260807 15:14:57.334849 3806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:57.341224 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 3806
I20260807 15:14:57.341331 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:57.341729 32667 external_mini_cluster.cc:1428] Setting key 7ca0204caf3530ef7168749d9a1b25d0
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115697, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:57.349833 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.336794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:56047 (local address 127.31.230.254:33181)
0807 15:14:57.336943 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:57.336947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:57.337978 (+ 1031us) server_negotiation.cc:407] Connection header received
0807 15:14:57.339461 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:57.339465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:57.339535 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:57.339612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:57.340812 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.341485 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.342746 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.342998 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.343124 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.345723 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:57.345729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:57.345733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:57.345775 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:57.347651 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.348175 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.348182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.348186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.348304 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.348665 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.348670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.348671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.348873 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:57.348987 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:57.349488 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 15:14:57.349645 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:14:57.350679 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.337187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33181 (local address 127.31.230.193:56047)
0807 15:14:57.337700 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:57.337754 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:57.339209 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:57.339797 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:57.339804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:57.340172 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:57.340658 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:57.340673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.341658 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.341661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:57.342612 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:57.342623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.343154 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.343159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:57.343339 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.343960 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:57.343978 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:57.345474 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:57.347778 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:57.347783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:57.347797 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:57.348060 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:57.348408 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:57.348410 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:57.348411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:57.348544 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:57.348986 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:57.348993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:57.349249 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 15:14:57.349447 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 15:14:57.349806 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":123,"threads_started":1}
I20260807 15:14:57.352137 3986 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33181
I20260807 15:14:57.352401 3986 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:57.352916 3986 heartbeater.cc:511] Master 127.31.230.254:33181 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:14:57.354650 3710 ts_manager.cc:194] Registered new tserver with Master: 43f73a84224b4a76b838638457f49633 (127.31.230.193:43433)
I20260807 15:14:57.355942 3710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:56047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:14:57.373402 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33181
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:57.488173 3992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:57.488453 3992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:57.488499 3992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:57.492604 3992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:57.492671 3992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:57.492730 3992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:14:57.498150 3992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:57.499284 3992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:57.500370 3992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:57.510972 4003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:57.510972 4006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:57.510972 4004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:57.511441 3992 server_base.cc:1034] running on GCE node
I20260807 15:14:57.511911 3992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:57.512564 3992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:57.513763 3992 hybrid_clock.cc:648] HybridClock initialized: now 1786115697513750 us; error 34 us; skew 500 ppm
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115697, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:57.516672 3992 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:14:57.517701 3992 webserver.cc:460] Webserver started at http://127.31.230.194:42429/ using document root <none> and password file <none>
I20260807 15:14:57.518292 3992 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:57.518388 3992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:57.518684 3992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:57.520859 3992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "0a919d4384a34dc99186c24fe41bcef3"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "cae611718f45cbcc1e4263f7c69c2781"
server_key_iv: "ba34e532d74f562a33fad0d2980fcd47"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.521454 3992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "0a919d4384a34dc99186c24fe41bcef3"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "cae611718f45cbcc1e4263f7c69c2781"
server_key_iv: "ba34e532d74f562a33fad0d2980fcd47"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.525735 3992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:57.528438 4015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.530290 3992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:57.530440 3992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "0a919d4384a34dc99186c24fe41bcef3"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "cae611718f45cbcc1e4263f7c69c2781"
server_key_iv: "ba34e532d74f562a33fad0d2980fcd47"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.530565 3992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:57.550235 3992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:57.553798 3992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:57.554023 3992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:57.554729 3992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:57.555620 3992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:57.555694 3992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.555768 3992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:57.555824 3992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.565615 3992 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:38703
I20260807 15:14:57.565629 4169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:38703 every 8 connection(s)
I20260807 15:14:57.566563 3992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:57.571614 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 3992
I20260807 15:14:57.571748 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:57.572046 32667 external_mini_cluster.cc:1428] Setting key e0cc3b5ba56fe1e6346849ddecb60dab
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115697, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:57.582353 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.568482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:39141 (local address 127.31.230.254:33181)
0807 15:14:57.568733 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:57.568737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:57.569506 (+ 769us) server_negotiation.cc:407] Connection header received
0807 15:14:57.571004 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:57.571007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:57.571068 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:57.571180 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:57.572572 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.573122 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.574602 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.575012 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.575142 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.578165 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:57.578172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:57.578178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:57.578212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:57.579957 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.580503 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.580509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.580512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.580628 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.580973 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.580978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.580982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.581275 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:57.581408 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:57.581989 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 15:14:57.582148 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260807 15:14:57.583056 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.568866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33181 (local address 127.31.230.194:39141)
0807 15:14:57.569342 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:57.569387 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:57.570784 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:57.571367 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:57.571374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:57.571787 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:57.572387 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:57.572402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.573267 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.573274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:57.574434 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:57.574445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.575219 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.575222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:57.575421 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.576146 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:57.576173 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:57.577924 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:57.580072 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:57.580077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:57.580119 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:57.580373 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:57.580730 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:57.580732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:57.580733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:57.580863 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:57.581390 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:57.581394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:57.581698 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 15:14:57.581866 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 15:14:57.582136 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:14:57.584364 4170 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33181
I20260807 15:14:57.584640 4170 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:57.585187 4170 heartbeater.cc:511] Master 127.31.230.254:33181 requested a full tablet report, sending...
I20260807 15:14:57.586431 3710 ts_manager.cc:194] Registered new tserver with Master: 0a919d4384a34dc99186c24fe41bcef3 (127.31.230.194:38703)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:14:57.587157 3710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:39141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:14:57.604801 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33181
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:57.721616 4179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:57.721909 4179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:57.721990 4179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:57.726516 4179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:57.726583 4179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:57.726673 4179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:14:57.732084 4179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:57.733201 4179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:57.734375 4179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:57.744643 4184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:57.744643 4187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:57.744668 4185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:57.745337 4179 server_base.cc:1034] running on GCE node
I20260807 15:14:57.745831 4179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:57.746482 4179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:57.747680 4179 hybrid_clock.cc:648] HybridClock initialized: now 1786115697747668 us; error 36 us; skew 500 ppm
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115697, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:57.750551 4179 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:14:57.751529 4179 webserver.cc:460] Webserver started at http://127.31.230.195:36515/ using document root <none> and password file <none>
I20260807 15:14:57.752064 4179 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:57.752123 4179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:57.752305 4179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:57.754485 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "e728464d2b3a4346af2ce67d50c79628"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "47a1ca23c1aae8ca04f5d21e7528a04a"
server_key_iv: "18104acaab5e1316e9f8bc44bb17400c"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.755010 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "e728464d2b3a4346af2ce67d50c79628"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "47a1ca23c1aae8ca04f5d21e7528a04a"
server_key_iv: "18104acaab5e1316e9f8bc44bb17400c"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.758857 4179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:57.761390 4201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.762754 4179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:57.762885 4179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "e728464d2b3a4346af2ce67d50c79628"
format_stamp: "Formatted at 2026-08-07 15:14:57 on dist-test-slave-gkw7"
server_key: "47a1ca23c1aae8ca04f5d21e7528a04a"
server_key_iv: "18104acaab5e1316e9f8bc44bb17400c"
server_key_version: "encryptionkey@0"
I20260807 15:14:57.763001 4179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:57.775149 4179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:57.778281 4179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:57.778487 4179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:57.779078 4179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:57.779914 4179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:57.779961 4179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.780030 4179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:57.780067 4179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:57.789335 4179 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:38963
I20260807 15:14:57.789357 4364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:38963 every 8 connection(s)
I20260807 15:14:57.790289 4179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:57.792338 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 4179
I20260807 15:14:57.792467 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:57.792779 32667 external_mini_cluster.cc:1428] Setting key 6d8be009eb80c2e02edff8345f028a60
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[3648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115697, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:57.805833 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.792236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:41755 (local address 127.31.230.254:33181)
0807 15:14:57.792426 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:57.792432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:57.793578 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 15:14:57.795420 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:57.795424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:57.795491 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:57.795620 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:57.796971 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.797688 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.799039 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.799320 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.799455 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.802444 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:57.802447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:57.802449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:57.802475 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:57.803715 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.804191 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.804193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.804194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.804282 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:57.804671 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:57.804673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:57.804675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:57.804849 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:57.804926 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:57.805482 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 15:14:57.805650 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:14:57.806743 4367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:57.792662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33181 (local address 127.31.230.195:41755)
0807 15:14:57.793346 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:57.793429 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:57.795126 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:57.795822 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:57.795830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:57.796250 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:57.796793 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:57.796809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.797850 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.797854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:57.798890 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:57.798900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:57.799511 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:57.799516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:57.799734 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:57.800479 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:57.800498 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:57.802269 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:57.803822 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:57.803827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:57.803840 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:57.804103 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:57.804391 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:57.804394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:57.804395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:57.804558 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:57.804948 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:57.804954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:57.805201 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 15:14:57.805420 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 15:14:57.805809 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":151,"threads_started":1}
I20260807 15:14:57.807921 4365 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33181
I20260807 15:14:57.808202 4365 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:57.808663 4365 heartbeater.cc:511] Master 127.31.230.254:33181 requested a full tablet report, sending...
I20260807 15:14:57.809834 3710 ts_manager.cc:194] Registered new tserver with Master: e728464d2b3a4346af2ce67d50c79628 (127.31.230.195:38963)
I20260807 15:14:57.810423 3710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:41755
I20260807 15:14:57.819273 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:14:57.836593 3767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:57.834736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37274 (local address 127.31.230.254:33181)
0807 15:14:57.834904 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:57.834910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:57.835142 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:14:57.835319 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:57.835323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:57.835378 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:57.835472 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:57.836415 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37274: BlockingRecv error: recv got EOF from 127.0.0.1:37274 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:14:57.837725 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 3806
I20260807 15:14:57.854314 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 3992
I20260807 15:14:57.865128 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 4179
I20260807 15:14:57.875216 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 3667
2026-08-07T15:14:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2471 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[4379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[4379](info): set up 2 sockets
Aug 07 15:14:57 dist-test-slave-gkw7 krb5kdc[4379](info): commencing operation
krb5kdc: starting...
W20260807 15:14:59.168459 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:14:59 dist-test-slave-gkw7 krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:14:59.312307 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37197
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:59.431547 4398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:59.431917 4398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:59.432034 4398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:59.436337 4398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:59.436457 4398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:59.436497 4398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:59.436558 4398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:59.436609 4398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:59.442500 4398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:59.443771 4398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:59.444965 4398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:59.454385 4408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:59.454363 4412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:59.454442 4398 server_base.cc:1034] running on GCE node
W20260807 15:14:59.454363 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:59.455325 4398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:59.456512 4398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:59.457707 4398 hybrid_clock.cc:648] HybridClock initialized: now 1786115699457671 us; error 48 us; skew 500 ppm
Aug 07 15:14:59 dist-test-slave-gkw7 krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115699, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:59.460952 4398 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:14:59.462044 4398 webserver.cc:460] Webserver started at http://127.31.230.254:40855/ using document root <none> and password file <none>
I20260807 15:14:59.462631 4398 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:59.462694 4398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:59.462929 4398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:59.465313 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28"
format_stamp: "Formatted at 2026-08-07 15:14:59 on dist-test-slave-gkw7"
server_key: "50a06a72cbe65559ae1af50fbaa3ace6"
server_key_iv: "4d1cd847fe0e8c65254f754c25bd35d3"
server_key_version: "encryptionkey@0"
I20260807 15:14:59.466030 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28"
format_stamp: "Formatted at 2026-08-07 15:14:59 on dist-test-slave-gkw7"
server_key: "50a06a72cbe65559ae1af50fbaa3ace6"
server_key_iv: "4d1cd847fe0e8c65254f754c25bd35d3"
server_key_version: "encryptionkey@0"
I20260807 15:14:59.470146 4398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:59.472831 4422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:59.474395 4398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:59.474558 4398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28"
format_stamp: "Formatted at 2026-08-07 15:14:59 on dist-test-slave-gkw7"
server_key: "50a06a72cbe65559ae1af50fbaa3ace6"
server_key_iv: "4d1cd847fe0e8c65254f754c25bd35d3"
server_key_version: "encryptionkey@0"
I20260807 15:14:59.474697 4398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:59.494194 4398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:59.497691 4398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:59.497910 4398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:59.505949 4398 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:37197
I20260807 15:14:59.506258 4506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:37197 every 8 connection(s)
I20260807 15:14:59.507121 4398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:59.510074 4507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:59.510438 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 4398
I20260807 15:14:59.510560 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:14:59.510847 32667 external_mini_cluster.cc:1428] Setting key 7a8a4058e1cc7f738430df25908986cc
I20260807 15:14:59.517243 4507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28: Bootstrap starting.
I20260807 15:14:59.520228 4507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28: Neither blocks nor log segments found. Creating new log.
Aug 07 15:14:59 dist-test-slave-gkw7 krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:14:59.521101 4507 log.cc:824] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:59.523205 4507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28: No bootstrap required, opened a new log
I20260807 15:14:59.526440 4507 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:14:59.526647 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:59.512305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.31.230.254:37197)
0807 15:14:59.512752 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:59.512768 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:59.512804 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:14:59.513476 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:59.513493 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:59.513909 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:59.514201 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:59.515215 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:59.516190 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:59.517397 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:59.517968 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:59.518098 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:59.521165 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:59.521172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:59.521187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:59.521244 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:59.523167 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:59.523668 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:59.523673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:59.523678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:59.523796 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:59.524112 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:59.524144 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:59.524145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:59.524564 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:59.525392 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:59.525574 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 15:14:59.525852 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
I20260807 15:14:59.526726 4507 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:59.526793 4507 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fe11c5d2514b0ab264e9f4dccf7a28, State: Initialized, Role: FOLLOWER
I20260807 15:14:59.527319 4507 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:14:59.527468 4507 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:59.527520 4507 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:59.527614 4507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:59.528631 4507 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:14:59.528999 4507 leader_election.cc:302] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4fe11c5d2514b0ab264e9f4dccf7a28; no voters:
I20260807 15:14:59.529428 4507 leader_election.cc:288] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:59.529567 4512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:59.529834 4512 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [term 1 LEADER]: Becoming Leader. State: Replica: e4fe11c5d2514b0ab264e9f4dccf7a28, State: Running, Role: LEADER
I20260807 15:14:59.530045 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:59.530272 4512 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:14:59.530875 4507 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:59.531432 4512 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4fe11c5d2514b0ab264e9f4dccf7a28. Latest consensus state: current_term: 1 leader_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } } }
I20260807 15:14:59.531643 4512 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:59.532430 4513 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fe11c5d2514b0ab264e9f4dccf7a28" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } } }
I20260807 15:14:59.532585 4513 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:59.533630 4520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:59.536696 4520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:59.542855 4520 catalog_manager.cc:1347] Generated new cluster ID: b92ef6b853a64b77b749611425b9d4f1
I20260807 15:14:59.542940 4520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:59.549052 4520 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:59.550506 4520 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:59.558795 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:59.553737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.31.230.254:37197)
0807 15:14:59.553877 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:59.553882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:59.554206 (+ 324us) server_negotiation.cc:407] Connection header received
0807 15:14:59.554384 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:59.554393 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:59.554440 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:59.554530 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:59.555075 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:59.555633 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:59.556523 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:59.556787 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:59.556903 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:59.557834 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:59.557838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:59.557840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:59.558329 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:59.558361 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:59.558378 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:59.558485 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:59.558570 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 15:14:59.558647 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":48}
I20260807 15:14:59.560139 4520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4fe11c5d2514b0ab264e9f4dccf7a28: Generated new TSK 0
I20260807 15:14:59.560890 4520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:59.569451 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 4398
2026-08-07T15:14:59Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1696 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:59 dist-test-slave-gkw7 krb5kdc[4559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:59 dist-test-slave-gkw7 krb5kdc[4559](info): set up 2 sockets
Aug 07 15:14:59 dist-test-slave-gkw7 krb5kdc[4559](info): commencing operation
krb5kdc: starting...
W20260807 15:15:00.831523 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:00 dist-test-slave-gkw7 krb5kdc[4559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:00.973335 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:46665
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:46665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:01.091449 4576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:01.091801 4576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:01.091886 4576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:01.095978 4576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:01.096084 4576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:01.096128 4576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:01.096167 4576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:01.096204 4576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:01.101835 4576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:46665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:46665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:01.103001 4576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:01.104179 4576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:01.113107 4585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.113190 4589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.113206 4583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:01.113299 4576 server_base.cc:1034] running on GCE node
I20260807 15:15:01.114112 4576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:01.115273 4576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:01.116452 4576 hybrid_clock.cc:648] HybridClock initialized: now 1786115701116421 us; error 42 us; skew 500 ppm
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:01.119421 4576 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:01.120389 4576 webserver.cc:460] Webserver started at http://127.31.230.254:45197/ using document root <none> and password file <none>
I20260807 15:15:01.120985 4576 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:01.121048 4576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:01.121272 4576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:01.123417 4576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "1e40e2faddeb44f5be083c74279671a3"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "b9bce3fc652c7aeedbc652d7f6d74d95"
server_key_iv: "bb1367d4391d9094e803a61d6895c05c"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.124017 4576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "1e40e2faddeb44f5be083c74279671a3"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "b9bce3fc652c7aeedbc652d7f6d74d95"
server_key_iv: "bb1367d4391d9094e803a61d6895c05c"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.127936 4576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:01.130465 4597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.131768 4576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:01.131917 4576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "1e40e2faddeb44f5be083c74279671a3"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "b9bce3fc652c7aeedbc652d7f6d74d95"
server_key_iv: "bb1367d4391d9094e803a61d6895c05c"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.132031 4576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:01.153929 4576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:01.157048 4576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:01.157275 4576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:01.164556 4576 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:46665
I20260807 15:15:01.164884 4677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:46665 every 8 connection(s)
I20260807 15:15:01.165619 4576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:01.168520 4678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:01.171700 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 4576
I20260807 15:15:01.171854 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:01.172148 32667 external_mini_cluster.cc:1428] Setting key 9396c9d64f0650c4f1ec78fddcfd67bf
I20260807 15:15:01.174922 4678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3: Bootstrap starting.
I20260807 15:15:01.177695 4678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:01.178682 4678 log.cc:824] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:01.181334 4678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3: No bootstrap required, opened a new log
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:01.184897 4678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e40e2faddeb44f5be083c74279671a3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 46665 } }
I20260807 15:15:01.185134 4678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:01.185170 4678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e40e2faddeb44f5be083c74279671a3, State: Initialized, Role: FOLLOWER
I20260807 15:15:01.185756 4678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e40e2faddeb44f5be083c74279671a3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 46665 } }
I20260807 15:15:01.185950 4678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:01.186026 4678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:01.186152 4678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:01.187321 4678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e40e2faddeb44f5be083c74279671a3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 46665 } }
I20260807 15:15:01.187717 4678 leader_election.cc:302] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e40e2faddeb44f5be083c74279671a3; no voters:
I20260807 15:15:01.188115 4678 leader_election.cc:288] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:01.188206 4687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:01.188413 4687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [term 1 LEADER]: Becoming Leader. State: Replica: 1e40e2faddeb44f5be083c74279671a3, State: Running, Role: LEADER
I20260807 15:15:01.188848 4687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e40e2faddeb44f5be083c74279671a3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 46665 } }
I20260807 15:15:01.189102 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.173722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.31.230.254:46665)
0807 15:15:01.174167 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:01.174190 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:01.174227 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:15:01.174833 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:01.174850 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:01.175247 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:01.175536 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:01.176519 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.177657 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.178930 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.179657 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.179783 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.182911 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:01.182916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:01.182931 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:01.182959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:01.185607 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.186142 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.186149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.186155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.186340 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.186670 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.186673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.186674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.187075 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:01.187216 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:01.187619 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 15:15:01.187972 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":160,"threads_started":1}
I20260807 15:15:01.189574 4678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:01.190691 4689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e40e2faddeb44f5be083c74279671a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e40e2faddeb44f5be083c74279671a3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 46665 } } }
I20260807 15:15:01.190835 4689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:01.191191 4688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e40e2faddeb44f5be083c74279671a3. Latest consensus state: current_term: 1 leader_uuid: "1e40e2faddeb44f5be083c74279671a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e40e2faddeb44f5be083c74279671a3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 46665 } } }
I20260807 15:15:01.191296 4688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:01.191562 4692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:01.194700 4692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:01.200635 4692 catalog_manager.cc:1347] Generated new cluster ID: 9ce04bd76c994c1c8da68928be5f752d
I20260807 15:15:01.200721 4692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:01.206166 4692 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:01.207414 4692 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:01.224035 4692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e40e2faddeb44f5be083c74279671a3: Generated new TSK 0
I20260807 15:15:01.224993 4692 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:15:01.258872 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:46665
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:01.382475 4718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:01.382781 4718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:01.382825 4718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:01.386971 4718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:01.387039 4718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:01.387104 4718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:01.392748 4718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:46665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:01.394006 4718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:01.395048 4718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:01.405722 4718 server_base.cc:1034] running on GCE node
W20260807 15:15:01.405682 4726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.405682 4727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.405694 4731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:01.406620 4718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:01.407281 4718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:01.408481 4718 hybrid_clock.cc:648] HybridClock initialized: now 1786115701408464 us; error 29 us; skew 500 ppm
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:01.412427 4718 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:15:01.413328 4718 webserver.cc:460] Webserver started at http://127.31.230.193:34215/ using document root <none> and password file <none>
I20260807 15:15:01.413959 4718 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:01.414026 4718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:01.414201 4718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:01.416275 4718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "71254ebb5d9440c7b7eabf1807db256f"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "abedcfe91a60cf3e9f79083ad919bb97"
server_key_iv: "78c2294f9e138e772745d54da468cd5d"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.416803 4718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "71254ebb5d9440c7b7eabf1807db256f"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "abedcfe91a60cf3e9f79083ad919bb97"
server_key_iv: "78c2294f9e138e772745d54da468cd5d"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.420632 4718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:15:01.423226 4740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.424543 4718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:01.424686 4718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "71254ebb5d9440c7b7eabf1807db256f"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "abedcfe91a60cf3e9f79083ad919bb97"
server_key_iv: "78c2294f9e138e772745d54da468cd5d"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.424810 4718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:01.440085 4718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:01.443486 4718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:01.443711 4718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:01.444355 4718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:01.445222 4718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:01.445304 4718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.445381 4718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:01.445426 4718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.455235 4718 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:41773
I20260807 15:15:01.455253 4898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:41773 every 8 connection(s)
I20260807 15:15:01.456254 4718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:01.457228 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 4718
I20260807 15:15:01.457367 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:01.457703 32667 external_mini_cluster.cc:1428] Setting key 81c7e5c3304ae514b5532210f33391bd
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:15:01.473153 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.458468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:55841 (local address 127.31.230.254:46665)
0807 15:15:01.458625 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:01.458629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:01.459817 (+ 1188us) server_negotiation.cc:407] Connection header received
0807 15:15:01.461993 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:01.461999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:01.462160 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:01.462280 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:01.463519 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.464434 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.465766 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.466244 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.466390 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.469298 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:01.469305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:01.469311 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:01.469342 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:01.470858 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.471457 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.471460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.471461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.471541 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.471855 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.471858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.471859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.472157 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:01.472290 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:01.472908 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 15:15:01.473021 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:15:01.474507 4904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.458981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:46665 (local address 127.31.230.193:55841)
0807 15:15:01.459616 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:01.459676 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:01.461707 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:01.462462 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:01.462469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:01.462855 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:01.463352 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:01.463367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.464569 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.464571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:01.465641 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:01.465651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.466436 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.466440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:01.466652 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.467326 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:01.467343 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:01.469110 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:01.470974 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:01.470979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:01.470993 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:01.471358 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:01.471623 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:01.471626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:01.471627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:01.471769 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:01.472284 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:01.472291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:01.472625 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 15:15:01.472861 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 15:15:01.473258 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":173,"threads_started":1}
I20260807 15:15:01.475873 4899 heartbeater.cc:347] Connected to a master server at 127.31.230.254:46665
I20260807 15:15:01.476163 4899 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:01.476804 4899 heartbeater.cc:511] Master 127.31.230.254:46665 requested a full tablet report, sending...
I20260807 15:15:01.478747 4623 ts_manager.cc:194] Registered new tserver with Master: 71254ebb5d9440c7b7eabf1807db256f (127.31.230.193:41773)
I20260807 15:15:01.480063 4623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:55841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:15:01.493072 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:46665
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:01.609462 4907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:01.609750 4907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:01.609839 4907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:01.613880 4907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:01.613945 4907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:01.614002 4907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:01.619271 4907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:46665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:01.620452 4907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:01.621503 4907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:01.632426 4917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.632418 4913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.632418 4915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:01.632527 4907 server_base.cc:1034] running on GCE node
I20260807 15:15:01.633111 4907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:01.633919 4907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:01.635191 4907 hybrid_clock.cc:648] HybridClock initialized: now 1786115701635126 us; error 51 us; skew 500 ppm
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:01.638499 4907 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:15:01.639477 4907 webserver.cc:460] Webserver started at http://127.31.230.194:34829/ using document root <none> and password file <none>
I20260807 15:15:01.640117 4907 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:01.640183 4907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:01.640444 4907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:01.642673 4907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "6c5d754eaec74e10893190ffdd2e1d08"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "f4b95734db021ba433178b2bbd268c3e"
server_key_iv: "5841220cb3db59b427ab914196bad731"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.643245 4907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "6c5d754eaec74e10893190ffdd2e1d08"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "f4b95734db021ba433178b2bbd268c3e"
server_key_iv: "5841220cb3db59b427ab914196bad731"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.647135 4907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:01.649668 4928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.651038 4907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:01.651211 4907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "6c5d754eaec74e10893190ffdd2e1d08"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "f4b95734db021ba433178b2bbd268c3e"
server_key_iv: "5841220cb3db59b427ab914196bad731"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.651332 4907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:01.679714 4907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:01.683405 4907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:01.683656 4907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:01.684346 4907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:01.685294 4907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:01.685374 4907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.685451 4907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:01.685496 4907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.695928 4907 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:33535
I20260807 15:15:01.695955 5080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:33535 every 8 connection(s)
I20260807 15:15:01.696887 4907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:01.701087 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 4907
I20260807 15:15:01.701222 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:01.701520 32667 external_mini_cluster.cc:1428] Setting key de937d1ef128318e193da101970ca614
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:15:01.713398 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.698997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:53261 (local address 127.31.230.254:46665)
0807 15:15:01.699178 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:01.699183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:01.700127 (+ 944us) server_negotiation.cc:407] Connection header received
0807 15:15:01.702047 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:01.702051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:01.702113 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:01.702209 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:01.703743 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.704320 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.706033 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.706470 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.706593 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.709267 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:01.709270 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:01.709272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:01.709297 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:01.710929 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.711603 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.711606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.711608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.711691 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.712130 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.712133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.712134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.712338 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:01.712418 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:01.713094 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 15:15:01.713268 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:15:01.714435 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.699480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:46665 (local address 127.31.230.194:53261)
0807 15:15:01.699961 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:01.700007 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:01.701665 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:01.702416 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:01.702425 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:01.702816 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:01.703558 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:01.703581 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.704463 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.704469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:01.705907 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:01.705918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.706642 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.706645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:01.706789 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.707435 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:01.707451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:01.709055 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:01.711051 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:01.711061 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:01.711084 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:01.711484 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:01.711795 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:01.711801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:01.711806 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:01.712000 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:01.712459 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:01.712468 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:01.712821 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 15:15:01.713042 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 15:15:01.713450 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:15:01.715788 5081 heartbeater.cc:347] Connected to a master server at 127.31.230.254:46665
I20260807 15:15:01.716025 5081 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:01.716539 5081 heartbeater.cc:511] Master 127.31.230.254:46665 requested a full tablet report, sending...
I20260807 15:15:01.717701 4623 ts_manager.cc:194] Registered new tserver with Master: 6c5d754eaec74e10893190ffdd2e1d08 (127.31.230.194:33535)
I20260807 15:15:01.718571 4623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:53261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:15:01.732492 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:46665
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:01.848659 5086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:01.848948 5086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:01.848994 5086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:01.853147 5086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:01.853230 5086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:01.853291 5086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:01.858898 5086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:46665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:01.859938 5086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:01.860934 5086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:01.870956 5086 server_base.cc:1034] running on GCE node
W20260807 15:15:01.870882 5092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.870908 5099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:01.870869 5093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:01.871529 5086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:01.872202 5086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:01.873383 5086 hybrid_clock.cc:648] HybridClock initialized: now 1786115701873358 us; error 38 us; skew 500 ppm
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:01.877085 5086 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:15:01.878031 5086 webserver.cc:460] Webserver started at http://127.31.230.195:45997/ using document root <none> and password file <none>
I20260807 15:15:01.878613 5086 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:01.878679 5086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:01.878858 5086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:01.881134 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "118db390dba9403b868a1968bb034e29"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "c17a84c78b1e967a5593d4bca11870fe"
server_key_iv: "72562fea42878b344fd5e28c29bf40ab"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.881762 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "118db390dba9403b868a1968bb034e29"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "c17a84c78b1e967a5593d4bca11870fe"
server_key_iv: "72562fea42878b344fd5e28c29bf40ab"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.885504 5086 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 15:15:01.888114 5109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.889959 5086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:01.890106 5086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "118db390dba9403b868a1968bb034e29"
format_stamp: "Formatted at 2026-08-07 15:15:01 on dist-test-slave-gkw7"
server_key: "c17a84c78b1e967a5593d4bca11870fe"
server_key_iv: "72562fea42878b344fd5e28c29bf40ab"
server_key_version: "encryptionkey@0"
I20260807 15:15:01.890231 5086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:01.905624 5086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:01.908972 5086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:01.909205 5086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:01.910043 5086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:01.910951 5086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:01.911000 5086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.911041 5086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:01.911056 5086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:01.920826 5086 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:43429
I20260807 15:15:01.920847 5261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:43429 every 8 connection(s)
I20260807 15:15:01.921885 5086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:01.930204 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 5086
I20260807 15:15:01.930374 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:01.930755 32667 external_mini_cluster.cc:1428] Setting key eb50aeeda134bc507fb9fe968b325ad4
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:01.937901 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.923889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:59273 (local address 127.31.230.254:46665)
0807 15:15:01.924034 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:01.924039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:01.924964 (+ 925us) server_negotiation.cc:407] Connection header received
0807 15:15:01.926549 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:01.926552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:01.926621 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:01.926730 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:01.928011 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.928722 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.930104 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.930429 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.930548 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.933813 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:01.933816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:01.933818 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:01.933844 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:01.935523 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.936123 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.936127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.936129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.936223 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.936628 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.936631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.936632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.936837 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:01.936955 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:01.937569 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 15:15:01.937712 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:15:01.938773 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.924283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:46665 (local address 127.31.230.195:59273)
0807 15:15:01.924796 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:01.924842 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:01.926305 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:01.926930 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:01.926937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:01.927332 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:01.927852 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:01.927866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.928858 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.928862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:01.929944 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:01.929959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.930652 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.930657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:01.930889 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.931621 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:01.931640 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:01.933645 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:01.935650 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:01.935656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:01.935672 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:01.935982 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:01.936326 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:01.936329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:01.936331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:01.936486 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:01.936956 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:01.936966 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:01.937268 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 15:15:01.937477 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 15:15:01.937875 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":122,"threads_started":1}
I20260807 15:15:01.940135 5262 heartbeater.cc:347] Connected to a master server at 127.31.230.254:46665
I20260807 15:15:01.940392 5262 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:01.940905 5262 heartbeater.cc:511] Master 127.31.230.254:46665 requested a full tablet report, sending...
I20260807 15:15:01.942092 4623 ts_manager.cc:194] Registered new tserver with Master: 118db390dba9403b868a1968bb034e29 (127.31.230.195:43429)
I20260807 15:15:01.942706 4623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:59273
I20260807 15:15:01.945420 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 15:15:01 dist-test-slave-gkw7 krb5kdc[4559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115701, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:01.978556 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:01.965172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.31.230.254:46665)
0807 15:15:01.965359 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:01.965364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:01.965540 (+ 176us) server_negotiation.cc:407] Connection header received
0807 15:15:01.965730 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:01.965734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:01.965791 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:01.965887 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:01.966653 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.967201 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.968069 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:01.968305 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:01.968424 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:01.974776 (+ 6352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:01.974780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:01.974783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:01.974814 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:01.976565 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.977003 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.977006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.977007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.977115 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:01.977403 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:01.977406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:01.977407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:01.977782 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:01.977908 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:01.978178 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 15:15:01.978335 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 15:15:01.978996 4623 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35454
I20260807 15:15:01.980085 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 4718
I20260807 15:15:01.994026 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 4907
I20260807 15:15:02.015025 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 5086
I20260807 15:15:02.028115 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 4576
2026-08-07T15:15:02Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2461 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:02 dist-test-slave-gkw7 krb5kdc[5279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:02 dist-test-slave-gkw7 krb5kdc[5279](info): set up 2 sockets
Aug 07 15:15:02 dist-test-slave-gkw7 krb5kdc[5279](info): commencing operation
krb5kdc: starting...
W20260807 15:15:03.333417 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:03 dist-test-slave-gkw7 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:03.469367 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:43095
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:03.588796 5302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:03.589088 5302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:03.589174 5302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:03.593204 5302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:03.593302 5302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:03.593338 5302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:03.593376 5302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:03.593407 5302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:03.598702 5302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:43095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:03.599805 5302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:03.600872 5302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:03.609853 5313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:03.609831 5309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:03.609836 5308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:03.609941 5302 server_base.cc:1034] running on GCE node
I20260807 15:15:03.610683 5302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:03.611804 5302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:03.613014 5302 hybrid_clock.cc:648] HybridClock initialized: now 1786115703612974 us; error 51 us; skew 500 ppm
Aug 07 15:15:03 dist-test-slave-gkw7 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115703, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:03.616024 5302 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:03.616981 5302 webserver.cc:460] Webserver started at http://127.31.230.254:41131/ using document root <none> and password file <none>
I20260807 15:15:03.617657 5302 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:03.617733 5302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:03.617990 5302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:03.620107 5302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "462f724578f545bca9290a335ba0c367"
format_stamp: "Formatted at 2026-08-07 15:15:03 on dist-test-slave-gkw7"
server_key: "bb3ae2306eaa538cdeab767680c501aa"
server_key_iv: "5320d032bcbd7ddaa02353a366a7b9a4"
server_key_version: "encryptionkey@0"
I20260807 15:15:03.620711 5302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "462f724578f545bca9290a335ba0c367"
format_stamp: "Formatted at 2026-08-07 15:15:03 on dist-test-slave-gkw7"
server_key: "bb3ae2306eaa538cdeab767680c501aa"
server_key_iv: "5320d032bcbd7ddaa02353a366a7b9a4"
server_key_version: "encryptionkey@0"
I20260807 15:15:03.624639 5302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:15:03.627187 5321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:03.628520 5302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:03.628695 5302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "462f724578f545bca9290a335ba0c367"
format_stamp: "Formatted at 2026-08-07 15:15:03 on dist-test-slave-gkw7"
server_key: "bb3ae2306eaa538cdeab767680c501aa"
server_key_iv: "5320d032bcbd7ddaa02353a366a7b9a4"
server_key_version: "encryptionkey@0"
I20260807 15:15:03.628816 5302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:03.656229 5302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:03.659484 5302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:03.659716 5302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:03.667140 5302 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:43095
I20260807 15:15:03.667222 5394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:43095 every 8 connection(s)
I20260807 15:15:03.668135 5302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:03.671018 5397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:03.676753 5397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367: Bootstrap starting.
I20260807 15:15:03.677004 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 5302
I20260807 15:15:03.677140 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:03.677449 32667 external_mini_cluster.cc:1428] Setting key 9110c81a448079a6f4815c5caaef2b80
I20260807 15:15:03.679610 5397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:03.680384 5397 log.cc:824] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:03.682726 5397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367: No bootstrap required, opened a new log
Aug 07 15:15:03 dist-test-slave-gkw7 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:03.687758 5397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462f724578f545bca9290a335ba0c367" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43095 } }
I20260807 15:15:03.688031 5397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:03.688117 5397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462f724578f545bca9290a335ba0c367, State: Initialized, Role: FOLLOWER
I20260807 15:15:03.688632 5397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462f724578f545bca9290a335ba0c367" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43095 } }
I20260807 15:15:03.688812 5397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:03.688912 5397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:03.689030 5397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:03.690138 5397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462f724578f545bca9290a335ba0c367" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43095 } }
I20260807 15:15:03.690516 5397 leader_election.cc:302] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 462f724578f545bca9290a335ba0c367; no voters:
I20260807 15:15:03.690927 5397 leader_election.cc:288] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:03.691054 5403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:03.691268 5403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [term 1 LEADER]: Becoming Leader. State: Replica: 462f724578f545bca9290a335ba0c367, State: Running, Role: LEADER
I20260807 15:15:03.691628 5403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462f724578f545bca9290a335ba0c367" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43095 } }
I20260807 15:15:03.692343 5397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:03.692919 5403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 462f724578f545bca9290a335ba0c367. Latest consensus state: current_term: 1 leader_uuid: "462f724578f545bca9290a335ba0c367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462f724578f545bca9290a335ba0c367" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43095 } } }
I20260807 15:15:03.693712 5403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:03.693600 5404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "462f724578f545bca9290a335ba0c367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462f724578f545bca9290a335ba0c367" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43095 } } }
I20260807 15:15:03.694013 5404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:03.694024 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:03.678946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.31.230.254:43095)
0807 15:15:03.679412 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:03.679427 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:03.679466 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:15:03.680185 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:03.680206 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:03.680614 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:03.680993 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:03.681900 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:03.683006 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:03.684257 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:03.684760 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:03.684890 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:03.688075 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:03.688082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:03.688110 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:03.688147 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:03.690144 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:03.690609 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:03.690615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:03.690623 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:03.690787 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:03.691168 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:03.691171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:03.691173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:03.691747 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:03.691961 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:03.692375 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 15:15:03.692976 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":129,"threads_started":1}
I20260807 15:15:03.694905 5417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:03.698287 5417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:03.704209 5417 catalog_manager.cc:1347] Generated new cluster ID: 9b9a1873ab3348869d410d37ab70e8d4
I20260807 15:15:03.704376 5417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:03.718262 5417 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:03.719403 5417 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:03.725579 5417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 462f724578f545bca9290a335ba0c367: Generated new TSK 0
I20260807 15:15:03.726511 5417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:15:03.765190 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43095
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:03.904909 5434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:03.905201 5434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:03.905248 5434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:03.909415 5434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:03.909483 5434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:03.909613 5434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:03.915284 5434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:03.916419 5434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:03.917490 5434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:03.927316 5449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:03.927445 5434 server_base.cc:1034] running on GCE node
W20260807 15:15:03.927347 5442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:03.927316 5446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:03.928076 5434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:03.928755 5434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:03.929941 5434 hybrid_clock.cc:648] HybridClock initialized: now 1786115703929934 us; error 39 us; skew 500 ppm
Aug 07 15:15:03 dist-test-slave-gkw7 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115703, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:03.932942 5434 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:15:03.934002 5434 webserver.cc:460] Webserver started at http://127.31.230.193:46431/ using document root <none> and password file <none>
I20260807 15:15:03.934602 5434 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:03.934695 5434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:03.934984 5434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:03.937103 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "186b3747565e4e88940473d633dde414"
format_stamp: "Formatted at 2026-08-07 15:15:03 on dist-test-slave-gkw7"
server_key: "58f2e73f53fff58995b576b8df531ff6"
server_key_iv: "19586c750ac66d7cd5ae6e097c184809"
server_key_version: "encryptionkey@0"
I20260807 15:15:03.937757 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "186b3747565e4e88940473d633dde414"
format_stamp: "Formatted at 2026-08-07 15:15:03 on dist-test-slave-gkw7"
server_key: "58f2e73f53fff58995b576b8df531ff6"
server_key_iv: "19586c750ac66d7cd5ae6e097c184809"
server_key_version: "encryptionkey@0"
I20260807 15:15:03.941785 5434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:03.944265 5457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:03.945724 5434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:03.945874 5434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "186b3747565e4e88940473d633dde414"
format_stamp: "Formatted at 2026-08-07 15:15:03 on dist-test-slave-gkw7"
server_key: "58f2e73f53fff58995b576b8df531ff6"
server_key_iv: "19586c750ac66d7cd5ae6e097c184809"
server_key_version: "encryptionkey@0"
I20260807 15:15:03.946003 5434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:03.981904 5434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:03.985142 5434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:03.985386 5434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:03.986141 5434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:03.987088 5434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:03.987167 5434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:03.987249 5434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:03.987308 5434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:03.997501 5434 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:42059
I20260807 15:15:03.997534 5617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:42059 every 8 connection(s)
I20260807 15:15:03.998610 5434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:04.003719 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 5434
I20260807 15:15:04.003867 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:04.004230 32667 external_mini_cluster.cc:1428] Setting key 72d8cd1579d5dfa3bf9f5c92f57935dc
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115703, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:04.015286 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.000628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:59221 (local address 127.31.230.254:43095)
0807 15:15:04.000820 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:04.000826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:04.001767 (+ 941us) server_negotiation.cc:407] Connection header received
0807 15:15:04.003204 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:04.003206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:04.003254 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:04.003332 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:04.005170 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.005856 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.007306 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.007619 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.007756 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.010603 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:04.010610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:04.010617 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:04.010655 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:04.012793 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.013353 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.013357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.013359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.013476 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.013909 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.013915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.013918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.014231 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:04.014360 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:04.014883 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 15:15:04.015048 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:15:04.016088 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.001021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43095 (local address 127.31.230.193:59221)
0807 15:15:04.001515 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:04.001637 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:04.002952 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:04.003698 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:04.003710 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:04.004307 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:04.004980 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:04.004996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.006017 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.006020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:04.007157 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:04.007168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.007822 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.007829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:04.008012 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.008660 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:04.008677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:04.010389 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:04.012930 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:04.012936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:04.012951 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:04.013223 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:04.013645 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:04.013647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:04.013648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:04.013794 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:04.014355 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:04.014361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:04.014616 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 15:15:04.014787 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 15:15:04.015063 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":124,"threads_started":1}
I20260807 15:15:04.017697 5619 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43095
I20260807 15:15:04.017990 5619 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:04.018577 5619 heartbeater.cc:511] Master 127.31.230.254:43095 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:15:04.020417 5344 ts_manager.cc:194] Registered new tserver with Master: 186b3747565e4e88940473d633dde414 (127.31.230.193:42059)
I20260807 15:15:04.021961 5344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:59221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:15:04.038256 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43095
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:04.154472 5628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:04.154754 5628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:04.154798 5628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:04.158896 5628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:04.158975 5628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:04.159076 5628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:04.164708 5628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:04.165899 5628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:04.166988 5628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:04.177104 5638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:04.177155 5628 server_base.cc:1034] running on GCE node
W20260807 15:15:04.177171 5636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:04.177158 5634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:04.177824 5628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:04.178481 5628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:04.179718 5628 hybrid_clock.cc:648] HybridClock initialized: now 1786115704179699 us; error 31 us; skew 500 ppm
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:04.183185 5628 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:15:04.184154 5628 webserver.cc:460] Webserver started at http://127.31.230.194:37567/ using document root <none> and password file <none>
I20260807 15:15:04.184757 5628 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:04.184854 5628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:04.185103 5628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:04.187269 5628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "dc6565d3d8ac4f3c86ec8decb50cde7e"
format_stamp: "Formatted at 2026-08-07 15:15:04 on dist-test-slave-gkw7"
server_key: "e3672c4c3a2f0236b40321258f877f29"
server_key_iv: "b135a9b9b89cada9574e12bc44e787b2"
server_key_version: "encryptionkey@0"
I20260807 15:15:04.187873 5628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "dc6565d3d8ac4f3c86ec8decb50cde7e"
format_stamp: "Formatted at 2026-08-07 15:15:04 on dist-test-slave-gkw7"
server_key: "e3672c4c3a2f0236b40321258f877f29"
server_key_iv: "b135a9b9b89cada9574e12bc44e787b2"
server_key_version: "encryptionkey@0"
I20260807 15:15:04.191843 5628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 15:15:04.194711 5646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:04.196204 5628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:04.196367 5628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "dc6565d3d8ac4f3c86ec8decb50cde7e"
format_stamp: "Formatted at 2026-08-07 15:15:04 on dist-test-slave-gkw7"
server_key: "e3672c4c3a2f0236b40321258f877f29"
server_key_iv: "b135a9b9b89cada9574e12bc44e787b2"
server_key_version: "encryptionkey@0"
I20260807 15:15:04.196499 5628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:04.210069 5628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:04.213016 5628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:04.213232 5628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:04.213936 5628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:04.214790 5628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:04.214865 5628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:04.214937 5628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:04.214989 5628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:04.224031 5628 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:36277
I20260807 15:15:04.224056 5801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:36277 every 8 connection(s)
I20260807 15:15:04.225051 5628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:04.226135 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 5628
I20260807 15:15:04.226266 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:04.226547 32667 external_mini_cluster.cc:1428] Setting key c94d06661005281c9e290b0fa5ad5503
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:15:04.241927 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.227171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:55331 (local address 127.31.230.254:43095)
0807 15:15:04.227341 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:04.227346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:04.228445 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 15:15:04.230619 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:04.230624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:04.230695 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:04.230817 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:04.232032 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.232952 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.234371 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.234815 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.234957 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.237845 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:04.237849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:04.237852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:04.237887 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:04.239500 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.240089 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.240092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.240093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.240174 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.240525 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.240527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.240528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.240715 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:04.240812 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:04.241443 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 15:15:04.241641 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:15:04.242901 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.227662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43095 (local address 127.31.230.194:55331)
0807 15:15:04.228261 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:04.228322 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:04.230338 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:04.230984 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:04.230991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:04.231371 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:04.231884 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:04.231897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.233115 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.233118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:04.234238 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:04.234249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.234978 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.234981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:04.235136 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.235833 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:04.235849 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:04.237639 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:04.239654 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:04.239660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:04.239676 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:04.239974 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:04.240267 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:04.240269 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:04.240271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:04.240428 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:04.240868 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:04.240875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:04.241158 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 15:15:04.241360 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 15:15:04.241855 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":180,"threads_started":1}
I20260807 15:15:04.244478 5803 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43095
I20260807 15:15:04.244751 5803 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:04.245298 5803 heartbeater.cc:511] Master 127.31.230.254:43095 requested a full tablet report, sending...
I20260807 15:15:04.246558 5344 ts_manager.cc:194] Registered new tserver with Master: dc6565d3d8ac4f3c86ec8decb50cde7e (127.31.230.194:36277)
I20260807 15:15:04.247197 5344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:55331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:15:04.261626 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43095
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:04.377781 5809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:04.378039 5809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:04.378082 5809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:04.382200 5809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:04.382290 5809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:04.382395 5809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:04.387701 5809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:04.388778 5809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:04.389967 5809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:04.400003 5819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:04.400024 5816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:04.400177 5809 server_base.cc:1034] running on GCE node
W20260807 15:15:04.400043 5815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:04.400651 5809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:04.401393 5809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:04.402547 5809 hybrid_clock.cc:648] HybridClock initialized: now 1786115704402508 us; error 51 us; skew 500 ppm
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:04.405511 5809 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:15:04.406499 5809 webserver.cc:460] Webserver started at http://127.31.230.195:43675/ using document root <none> and password file <none>
I20260807 15:15:04.407047 5809 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:04.407106 5809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:04.407287 5809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:04.409329 5809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "c7a40abc47354e7a9ddd36376959858e"
format_stamp: "Formatted at 2026-08-07 15:15:04 on dist-test-slave-gkw7"
server_key: "57759f98d36ae5b800e12a39b7d413d4"
server_key_iv: "91ce3791f062dc134ab159ba16416e5b"
server_key_version: "encryptionkey@0"
I20260807 15:15:04.409938 5809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "c7a40abc47354e7a9ddd36376959858e"
format_stamp: "Formatted at 2026-08-07 15:15:04 on dist-test-slave-gkw7"
server_key: "57759f98d36ae5b800e12a39b7d413d4"
server_key_iv: "91ce3791f062dc134ab159ba16416e5b"
server_key_version: "encryptionkey@0"
I20260807 15:15:04.413870 5809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:04.416332 5829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:04.417644 5809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:04.417778 5809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "c7a40abc47354e7a9ddd36376959858e"
format_stamp: "Formatted at 2026-08-07 15:15:04 on dist-test-slave-gkw7"
server_key: "57759f98d36ae5b800e12a39b7d413d4"
server_key_iv: "91ce3791f062dc134ab159ba16416e5b"
server_key_version: "encryptionkey@0"
I20260807 15:15:04.417891 5809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:04.431521 5809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:04.435217 5809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:04.435402 5809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:04.436040 5809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:04.436939 5809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:04.437014 5809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:04.437088 5809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:04.437124 5809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:04.447099 5809 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:38159
I20260807 15:15:04.447118 6002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:38159 every 8 connection(s)
I20260807 15:15:04.448066 5809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:04.449642 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 5809
I20260807 15:15:04.449807 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:04.450198 32667 external_mini_cluster.cc:1428] Setting key 7d5fb5b2f940cf922acb00139dfe39fe
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:04.462466 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.450176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:37693 (local address 127.31.230.254:43095)
0807 15:15:04.450305 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:04.450315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:04.451491 (+ 1176us) server_negotiation.cc:407] Connection header received
0807 15:15:04.453259 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:04.453262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:04.453309 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:04.453377 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:04.454585 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.455120 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.456321 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.456537 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.456615 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.459211 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:04.459214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:04.459217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:04.459241 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:04.460573 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.461047 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.461050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.461051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.461132 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.461437 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.461439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.461441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.461688 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:04.461788 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:04.462235 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 15:15:04.462339 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:15:04.463546 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.450629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43095 (local address 127.31.230.195:37693)
0807 15:15:04.451320 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:04.451380 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:04.453059 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:04.453517 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:04.453523 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:04.453961 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:04.454458 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:04.454472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.455314 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.455317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:04.456210 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:04.456220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.456669 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.456671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:04.456822 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.457500 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:04.457517 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:04.459048 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:04.460680 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:04.460687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:04.460702 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:04.460957 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:04.461219 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:04.461222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:04.461223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:04.461355 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:04.461788 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:04.461793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:04.462024 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 15:15:04.462215 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 15:15:04.462483 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":140,"threads_started":1}
I20260807 15:15:04.464751 6004 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43095
I20260807 15:15:04.465008 6004 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:04.465516 6004 heartbeater.cc:511] Master 127.31.230.254:43095 requested a full tablet report, sending...
I20260807 15:15:04.466629 5340 ts_manager.cc:194] Registered new tserver with Master: c7a40abc47354e7a9ddd36376959858e (127.31.230.195:38159)
I20260807 15:15:04.467255 5340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:37693
I20260807 15:15:04.476603 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:15:04.505014 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.495097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55222 (local address 127.31.230.193:42059)
0807 15:15:04.495471 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:04.495493 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:04.495523 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:15:04.495588 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:04.495593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:04.495806 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:04.495977 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:04.496720 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.497505 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.498433 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.498823 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.498942 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.501480 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:04.501489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:04.501503 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:04.501587 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:04.503123 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.503585 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.503598 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.503600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.503682 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.503974 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.503978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.503980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.504240 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:04.504497 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:04.504665 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 15:15:04.504829 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":125,"threads_started":1}
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:04.516623 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:04.508003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.31.230.254:43095)
0807 15:15:04.508145 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:04.508151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:04.508241 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:15:04.508425 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:04.508428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:04.508487 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:04.508573 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:04.509207 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.509850 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.510669 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:04.510911 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:04.511021 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:04.513445 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:04.513456 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:04.513459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:04.513490 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:04.514810 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.515233 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.515238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.515240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.515333 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:04.515647 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:04.515651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:04.515653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:04.515906 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:04.516057 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:04.516320 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 15:15:04.516451 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 15:15:04.517087 5340 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60082
I20260807 15:15:04.517833 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 5434
I20260807 15:15:04.528329 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 5628
I20260807 15:15:04.542086 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 5809
I20260807 15:15:04.552248 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 5302
2026-08-07T15:15:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2519 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[6018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[6018](info): set up 2 sockets
Aug 07 15:15:04 dist-test-slave-gkw7 krb5kdc[6018](info): commencing operation
krb5kdc: starting...
W20260807 15:15:05.814026 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:05 dist-test-slave-gkw7 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:05.948698 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33523
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:33523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:06.069360 6033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:06.069687 6033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:06.069784 6033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:06.073906 6033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:06.073972 6033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:06.073995 6033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:06.074057 6033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:06.074101 6033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:06.079555 6033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:33523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:06.080789 6033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:06.081965 6033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:06.090570 6041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.090592 6044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.090652 6040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:06.090728 6033 server_base.cc:1034] running on GCE node
I20260807 15:15:06.091485 6033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:06.092628 6033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:06.093842 6033 hybrid_clock.cc:648] HybridClock initialized: now 1786115706093807 us; error 46 us; skew 500 ppm
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:06.096959 6033 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:06.097803 6033 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:06.097872 6033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:06.098094 6033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:06.100193 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "3919d7297d384db5937e5b5fb9d859e4"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "31c71a2706c9cd89e9c541991c0a73bf"
server_key_iv: "82ce719a3b9f8f95b4fa97d5180aaa93"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.100826 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "3919d7297d384db5937e5b5fb9d859e4"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "31c71a2706c9cd89e9c541991c0a73bf"
server_key_iv: "82ce719a3b9f8f95b4fa97d5180aaa93"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.104799 6033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:15:06.107486 6059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.108843 6033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:15:06.108979 6033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "3919d7297d384db5937e5b5fb9d859e4"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "31c71a2706c9cd89e9c541991c0a73bf"
server_key_iv: "82ce719a3b9f8f95b4fa97d5180aaa93"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.109093 6033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:06.140338 6033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:06.143925 6033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:06.144150 6033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:06.151384 6033 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:33523
I20260807 15:15:06.151701 6136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:33523 every 8 connection(s)
I20260807 15:15:06.152112 6033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:06.155136 6137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:06.156912 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 6033
I20260807 15:15:06.157073 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:06.157351 32667 external_mini_cluster.cc:1428] Setting key 1bed300d2ce3e7a3c3ef6bb336205995
I20260807 15:15:06.162312 6137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4: Bootstrap starting.
I20260807 15:15:06.165016 6137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:06.165920 6137 log.cc:824] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:06.168311 6137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4: No bootstrap required, opened a new log
I20260807 15:15:06.173261 6137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919d7297d384db5937e5b5fb9d859e4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33523 } }
I20260807 15:15:06.173512 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.158896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.31.230.254:33523)
0807 15:15:06.159338 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.159357 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.159402 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:15:06.160252 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.160280 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.160724 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.161115 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:06.161945 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.163385 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.164320 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.165027 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.165198 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.168268 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:06.168275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:06.168292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:06.168324 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:06.170298 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.170737 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.170745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.170754 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.170928 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.171210 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.171214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.171217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.171779 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:06.171983 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.172181 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.172500 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260807 15:15:06.173560 6137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:06.173632 6137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3919d7297d384db5937e5b5fb9d859e4, State: Initialized, Role: FOLLOWER
I20260807 15:15:06.174113 6137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919d7297d384db5937e5b5fb9d859e4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33523 } }
I20260807 15:15:06.174283 6137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:06.174376 6137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:06.174496 6137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:06.175695 6137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919d7297d384db5937e5b5fb9d859e4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33523 } }
I20260807 15:15:06.176038 6137 leader_election.cc:302] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3919d7297d384db5937e5b5fb9d859e4; no voters:
I20260807 15:15:06.176364 6137 leader_election.cc:288] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:06.176470 6143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:06.176712 6143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [term 1 LEADER]: Becoming Leader. State: Replica: 3919d7297d384db5937e5b5fb9d859e4, State: Running, Role: LEADER
I20260807 15:15:06.177073 6143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919d7297d384db5937e5b5fb9d859e4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33523 } }
I20260807 15:15:06.177716 6137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:06.178906 6145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3919d7297d384db5937e5b5fb9d859e4. Latest consensus state: current_term: 1 leader_uuid: "3919d7297d384db5937e5b5fb9d859e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919d7297d384db5937e5b5fb9d859e4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33523 } } }
I20260807 15:15:06.179103 6145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:06.178905 6144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3919d7297d384db5937e5b5fb9d859e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919d7297d384db5937e5b5fb9d859e4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33523 } } }
I20260807 15:15:06.179306 6144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:06.180161 6155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:06.183466 6155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:06.188644 6155 catalog_manager.cc:1347] Generated new cluster ID: bf3e829e668f4322bd33587310056999
I20260807 15:15:06.188710 6155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:06.204535 6155 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:06.205705 6155 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:06.215535 6155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4: Generated new TSK 0
I20260807 15:15:06.216289 6155 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:15:06.246805 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33523
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:06.365178 6170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:06.365450 6170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:06.365556 6170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:06.369561 6170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:06.369622 6170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:06.369700 6170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:06.374951 6170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:06.376039 6170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:06.377101 6170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:06.387159 6179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.387219 6177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.387213 6175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:06.387260 6170 server_base.cc:1034] running on GCE node
I20260807 15:15:06.387830 6170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:06.388478 6170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:06.389655 6170 hybrid_clock.cc:648] HybridClock initialized: now 1786115706389638 us; error 33 us; skew 500 ppm
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:06.392530 6170 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:15:06.393299 6170 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:06.393397 6170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:06.393694 6170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:06.395929 6170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "7643d1fd02e5464baae74e23994b6dd9"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "1434ad74927d31964f9d2447c7423885"
server_key_iv: "360826e89dd519713bd2abc0b78d2e9b"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.396574 6170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "7643d1fd02e5464baae74e23994b6dd9"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "1434ad74927d31964f9d2447c7423885"
server_key_iv: "360826e89dd519713bd2abc0b78d2e9b"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.400453 6170 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:15:06.402931 6185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.404201 6170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:06.404361 6170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "7643d1fd02e5464baae74e23994b6dd9"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "1434ad74927d31964f9d2447c7423885"
server_key_iv: "360826e89dd519713bd2abc0b78d2e9b"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.404481 6170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:06.424757 6170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:06.428435 6170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:06.428687 6170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:06.429337 6170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:06.430343 6170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:06.430425 6170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.430507 6170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:06.430572 6170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.441408 6170 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:40105
I20260807 15:15:06.441475 6346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:40105 every 8 connection(s)
I20260807 15:15:06.442198 6170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:06.444757 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 6170
I20260807 15:15:06.444898 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:06.445243 32667 external_mini_cluster.cc:1428] Setting key 3e1e875eb8571bbc65b70e6ded6812af
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:15:06.457737 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.444276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:50315 (local address 127.31.230.254:33523)
0807 15:15:06.444455 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.444459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.445413 (+ 954us) server_negotiation.cc:407] Connection header received
0807 15:15:06.446826 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.446830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.446889 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.446992 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:06.448676 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.449298 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.450552 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.450823 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.450942 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.453585 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:06.453591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:06.453597 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:06.453623 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:06.455263 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.455942 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.455947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.455949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.456063 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.456465 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.456469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.456471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.456680 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:06.456823 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.457365 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.457508 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:15:06.458735 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.444655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33523 (local address 127.31.230.193:50315)
0807 15:15:06.445236 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:06.445301 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:06.446622 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:06.447198 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:06.447207 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:06.447764 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:06.448493 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.448518 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.449448 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.449458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.450416 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.450427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.450984 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.450989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.451174 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.451852 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:06.451868 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:06.453388 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:06.455422 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:06.455429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:06.455445 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:06.455824 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:06.456177 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:06.456181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:06.456183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:06.456345 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:06.456823 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:06.456831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:06.457102 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 15:15:06.457318 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 15:15:06.457720 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":194,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:15:06.460232 6347 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33523
I20260807 15:15:06.460510 6347 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:06.461117 6347 heartbeater.cc:511] Master 127.31.230.254:33523 requested a full tablet report, sending...
I20260807 15:15:06.462793 6083 ts_manager.cc:194] Registered new tserver with Master: 7643d1fd02e5464baae74e23994b6dd9 (127.31.230.193:40105)
I20260807 15:15:06.463972 6083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:50315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:15:06.477995 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33523
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:06.593596 6356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:06.593845 6356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:06.593928 6356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:06.598066 6356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:06.598121 6356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:06.598217 6356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:06.603427 6356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:06.604496 6356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:06.605567 6356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:06.616074 6364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.616147 6362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.616061 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:06.616257 6356 server_base.cc:1034] running on GCE node
I20260807 15:15:06.616786 6356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:06.617513 6356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:06.618739 6356 hybrid_clock.cc:648] HybridClock initialized: now 1786115706618693 us; error 59 us; skew 500 ppm
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:06.622043 6356 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:15:06.622814 6356 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:06.622913 6356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:06.623164 6356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:06.625320 6356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "97b3f7404e864d45a43c3c6e81019953"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "4dffbb3a7b9e707767be92bc504eefdd"
server_key_iv: "4e3eda1117c3a2875e68c310b48a48b1"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.625980 6356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "97b3f7404e864d45a43c3c6e81019953"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "4dffbb3a7b9e707767be92bc504eefdd"
server_key_iv: "4e3eda1117c3a2875e68c310b48a48b1"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.629779 6356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:06.632289 6375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.633615 6356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:06.633749 6356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "97b3f7404e864d45a43c3c6e81019953"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "4dffbb3a7b9e707767be92bc504eefdd"
server_key_iv: "4e3eda1117c3a2875e68c310b48a48b1"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.633864 6356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:06.644832 6356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:06.648166 6356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:06.648386 6356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:06.648983 6356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:06.649894 6356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:06.649969 6356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.650040 6356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:06.650091 6356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.659228 6356 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:41413
I20260807 15:15:06.659247 6551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:41413 every 8 connection(s)
I20260807 15:15:06.659894 6356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:06.666185 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 6356
I20260807 15:15:06.666333 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:06.666641 32667 external_mini_cluster.cc:1428] Setting key 67d5911051b45a5d4d94b8967a64c5f7
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:06.675853 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.661889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:51915 (local address 127.31.230.254:33523)
0807 15:15:06.662064 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.662067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.662996 (+ 929us) server_negotiation.cc:407] Connection header received
0807 15:15:06.664492 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.664495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.664557 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.664666 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:06.666084 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.666847 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.668263 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.668505 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.668626 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.671407 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:06.671414 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:06.671419 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:06.671453 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:06.673138 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.673953 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.673956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.673957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.674043 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.674516 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.674519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.674520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.674712 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:06.674808 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.675601 (+ 793us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.675715 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:15:06.677217 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.662299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33523 (local address 127.31.230.194:51915)
0807 15:15:06.662807 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:06.662854 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:06.664253 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:06.664854 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:06.664861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:06.665258 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:06.665904 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.665927 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.666994 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.666997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.668136 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.668146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.668671 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.668675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.668850 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.669620 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:06.669640 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:06.671225 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:06.673275 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:06.673285 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:06.673308 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:06.673814 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:06.674157 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:06.674163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:06.674167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:06.674387 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:06.674845 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:06.674855 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:06.675261 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 15:15:06.675551 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 15:15:06.675986 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:15:06.678737 6552 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33523
I20260807 15:15:06.678997 6552 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:06.679519 6552 heartbeater.cc:511] Master 127.31.230.254:33523 requested a full tablet report, sending...
I20260807 15:15:06.680729 6083 ts_manager.cc:194] Registered new tserver with Master: 97b3f7404e864d45a43c3c6e81019953 (127.31.230.194:41413)
I20260807 15:15:06.681360 6083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:51915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:15:06.699003 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33523
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:06.819025 6558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:06.819360 6558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:06.819445 6558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:06.823616 6558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:06.823725 6558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:06.823827 6558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:06.829357 6558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:06.830621 6558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:06.831758 6558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:06.841997 6570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:06.842038 6558 server_base.cc:1034] running on GCE node
W20260807 15:15:06.842027 6568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:06.841997 6574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:06.842741 6558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:06.843461 6558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:06.844659 6558 hybrid_clock.cc:648] HybridClock initialized: now 1786115706844637 us; error 37 us; skew 500 ppm
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:06.848124 6558 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:15:06.849159 6558 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:06.849232 6558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:06.849434 6558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:06.851698 6558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "35ad7d4796d04289b6aa08a3e2b42329"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "8aa5ff78126d1bba23f3d388d5f1ba2a"
server_key_iv: "c2e8408f1ca8bf27cb17d13f8c085faa"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.852288 6558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "35ad7d4796d04289b6aa08a3e2b42329"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "8aa5ff78126d1bba23f3d388d5f1ba2a"
server_key_iv: "c2e8408f1ca8bf27cb17d13f8c085faa"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.856550 6558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:15:06.859236 6581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.860695 6558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:06.860850 6558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "35ad7d4796d04289b6aa08a3e2b42329"
format_stamp: "Formatted at 2026-08-07 15:15:06 on dist-test-slave-gkw7"
server_key: "8aa5ff78126d1bba23f3d388d5f1ba2a"
server_key_iv: "c2e8408f1ca8bf27cb17d13f8c085faa"
server_key_version: "encryptionkey@0"
I20260807 15:15:06.860970 6558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:06.888685 6558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:06.892798 6558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:06.893033 6558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:06.893895 6558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:06.895138 6558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:06.895190 6558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.895234 6558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:06.895251 6558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:06.904953 6558 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:38445
I20260807 15:15:06.904973 6740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:38445 every 8 connection(s)
I20260807 15:15:06.905694 6558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:06.907269 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 6558
I20260807 15:15:06.907415 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:06.907765 32667 external_mini_cluster.cc:1428] Setting key a08fd5523847319009d9f9a2ffdb9000
Aug 07 15:15:06 dist-test-slave-gkw7 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115706, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:06.920496 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.909445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:43101 (local address 127.31.230.254:33523)
0807 15:15:06.909579 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.909583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.909598 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:06.910742 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.910744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.910789 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.910845 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:06.912123 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.912766 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.914056 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.914297 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.914382 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.916964 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:06.916966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:06.916968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:06.916996 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:06.918575 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.919062 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.919064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.919065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.919143 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.919451 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.919453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.919454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.919653 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:06.919730 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.920264 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.920367 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:15:06.921790 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.908318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33523 (local address 127.31.230.195:43101)
0807 15:15:06.908947 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:06.909011 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:06.910503 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:06.910996 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:06.911003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:06.911384 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:06.911963 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.911980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.912904 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.912907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.913946 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.913956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.914447 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.914450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.914597 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.915257 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:06.915275 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:06.916801 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:06.918685 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:06.918690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:06.918704 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:06.918974 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:06.919224 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:06.919226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:06.919227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:06.919372 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:06.919756 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:06.919762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:06.920011 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 15:15:06.920197 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 15:15:06.920563 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":152,"threads_started":1}
I20260807 15:15:06.923081 6741 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33523
I20260807 15:15:06.923352 6741 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:06.923851 6741 heartbeater.cc:511] Master 127.31.230.254:33523 requested a full tablet report, sending...
I20260807 15:15:06.924997 6083 ts_manager.cc:194] Registered new tserver with Master: 35ad7d4796d04289b6aa08a3e2b42329 (127.31.230.195:38445)
I20260807 15:15:06.925684 6083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:43101
I20260807 15:15:06.935092 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:06.944917 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.937274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.31.230.254:33523)
0807 15:15:06.937431 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.937440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.937708 (+ 268us) server_negotiation.cc:407] Connection header received
0807 15:15:06.937800 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.937802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.937847 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.937952 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:06.938747 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.939278 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.940093 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.940329 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.940406 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.941947 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:06.941949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:06.941951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:06.941973 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:06.943279 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.943688 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.943690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.943692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.943771 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:06.944056 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:06.944058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:06.944059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:06.944301 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:06.944404 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.944680 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.944786 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:15:06.948208 6083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:06.951447 6083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:06.968298 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.961250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.31.230.194:41413)
0807 15:15:06.962347 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.962362 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.963176 (+ 814us) server_negotiation.cc:407] Connection header received
0807 15:15:06.963282 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.963287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.963475 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.963621 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:06.963933 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.965231 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.966791 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.967484 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.967612 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.967747 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.967996 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.968114 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":122,"threads_started":1}
I20260807 15:15:06.968684 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.961120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41413 (local address 127.0.0.1:58214)
0807 15:15:06.963068 (+ 1948us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:06.963093 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:06.963166 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:06.963614 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:06.963617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:06.963634 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:06.963804 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.963810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.965377 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.965380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.966643 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.966653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.967631 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.967635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.967767 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.967813 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:15:06.967940 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:15:06.968073 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":105,"thread_start_us":72,"threads_started":1}
I20260807 15:15:06.969122 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.961717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.31.230.193:40105)
0807 15:15:06.962420 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.962428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.962820 (+ 392us) server_negotiation.cc:407] Connection header received
0807 15:15:06.963051 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.963056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.963308 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.963476 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:06.963983 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.965209 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.967224 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.968012 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.968129 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.968252 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.968864 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.968977 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":182,"threads_started":1}
I20260807 15:15:06.969156 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.961730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:38445 (local address 127.0.0.1:50330)
0807 15:15:06.962655 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:06.962719 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:06.962884 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:06.964070 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:06.964077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:06.964099 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:06.964202 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.964210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.966024 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.966028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.967285 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.967294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.968695 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.968702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.968825 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.968854 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:15:06.968934 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:15:06.969007 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":5,"spinlock_wait_cycles":820224,"thread_start_us":120,"threads_started":1}
I20260807 15:15:06.968763 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.961223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:40105 (local address 127.0.0.1:45804)
0807 15:15:06.962655 (+ 1432us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:06.962719 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:06.962884 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:06.963486 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:06.963492 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:06.963515 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:06.963795 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.963804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.965383 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.965386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.967070 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:06.967094 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.968327 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.968333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:06.968438 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.968472 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 15:15:06.968541 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:15:06.968607 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":149,"spinlock_wait_cycles":906240,"thread_start_us":126,"threads_started":1}
I20260807 15:15:06.969614 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:06.962205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.31.230.195:38445)
0807 15:15:06.962562 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:06.962577 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:06.962869 (+ 292us) server_negotiation.cc:407] Connection header received
0807 15:15:06.963692 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:06.963698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:06.963911 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:06.964074 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:06.964334 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.965874 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.967418 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:06.968316 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:06.968468 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:06.968617 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:06.969317 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 15:15:06.969419 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":101,"threads_started":1}
I20260807 15:15:06.972116 6651 tablet_service.cc:1467] Processing CreateTablet for tablet 3c85bd63cec84b78b365ea94072f5ab6 (DEFAULT_TABLE table=test-table [id=b49ba9f9054c4f5fa56f14ebcc6d0ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:06.972326 6254 tablet_service.cc:1467] Processing CreateTablet for tablet 3c85bd63cec84b78b365ea94072f5ab6 (DEFAULT_TABLE table=test-table [id=b49ba9f9054c4f5fa56f14ebcc6d0ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:06.972326 6450 tablet_service.cc:1467] Processing CreateTablet for tablet 3c85bd63cec84b78b365ea94072f5ab6 (DEFAULT_TABLE table=test-table [id=b49ba9f9054c4f5fa56f14ebcc6d0ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:06.973318 6651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c85bd63cec84b78b365ea94072f5ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:06.973480 6254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c85bd63cec84b78b365ea94072f5ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:06.973480 6450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c85bd63cec84b78b365ea94072f5ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:06.980394 6769 tablet_bootstrap.cc:492] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Bootstrap starting.
I20260807 15:15:06.981959 6771 tablet_bootstrap.cc:492] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Bootstrap starting.
I20260807 15:15:06.983204 6769 tablet_bootstrap.cc:654] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:06.984097 6771 tablet_bootstrap.cc:654] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:06.984727 6769 log.cc:824] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:06.985507 6771 log.cc:824] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:06.985987 6770 tablet_bootstrap.cc:492] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: Bootstrap starting.
I20260807 15:15:06.986887 6769 tablet_bootstrap.cc:492] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: No bootstrap required, opened a new log
I20260807 15:15:06.987107 6769 ts_tablet_manager.cc:1397] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 15:15:06.987808 6771 tablet_bootstrap.cc:492] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: No bootstrap required, opened a new log
I20260807 15:15:06.988026 6771 ts_tablet_manager.cc:1397] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:15:06.988478 6770 tablet_bootstrap.cc:654] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:06.989384 6770 log.cc:824] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:06.990430 6769 raft_consensus.cc:348] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:06.990696 6769 raft_consensus.cc:374] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:06.990751 6769 raft_consensus.cc:729] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ad7d4796d04289b6aa08a3e2b42329, State: Initialized, Role: FOLLOWER
I20260807 15:15:06.991302 6769 consensus_queue.cc:260] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:06.992105 6770 tablet_bootstrap.cc:492] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: No bootstrap required, opened a new log
I20260807 15:15:06.992326 6770 ts_tablet_manager.cc:1397] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:15:06.992401 6769 ts_tablet_manager.cc:1428] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:15:06.992388 6771 raft_consensus.cc:348] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:06.992702 6771 raft_consensus.cc:374] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:06.992759 6771 raft_consensus.cc:729] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b3f7404e864d45a43c3c6e81019953, State: Initialized, Role: FOLLOWER
I20260807 15:15:06.992789 6741 heartbeater.cc:503] Master 127.31.230.254:33523 was elected leader, sending a full tablet report...
I20260807 15:15:06.993257 6771 consensus_queue.cc:260] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:06.994012 6552 heartbeater.cc:503] Master 127.31.230.254:33523 was elected leader, sending a full tablet report...
I20260807 15:15:06.994772 6771 ts_tablet_manager.cc:1428] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:15:06.998039 6770 raft_consensus.cc:348] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:06.998275 6770 raft_consensus.cc:374] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:06.998312 6770 raft_consensus.cc:729] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7643d1fd02e5464baae74e23994b6dd9, State: Initialized, Role: FOLLOWER
I20260807 15:15:06.998710 6770 consensus_queue.cc:260] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:06.999365 6347 heartbeater.cc:503] Master 127.31.230.254:33523 was elected leader, sending a full tablet report...
I20260807 15:15:06.999729 6770 ts_tablet_manager.cc:1428] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:15:07.105767 6775 raft_consensus.cc:482] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:07.106046 6775 raft_consensus.cc:504] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:07.107441 6775 leader_election.cc:288] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b3f7404e864d45a43c3c6e81019953 (127.31.230.194:41413), 7643d1fd02e5464baae74e23994b6dd9 (127.31.230.193:40105)
I20260807 15:15:07.112968 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.107722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41413 (local address 127.31.230.195:49215)
0807 15:15:07.107939 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:07.107958 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:07.108071 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:07.108454 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:07.108457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:07.108477 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:07.108625 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:07.108632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.109875 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.109878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:07.111341 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:07.111352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.112356 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.112359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:07.112531 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:07.112558 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:15:07.112646 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:15:07.112784 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":74}
I20260807 15:15:07.112972 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.107895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:49215 (local address 127.31.230.194:41413)
0807 15:15:07.108195 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:07.108200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:07.108223 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:15:07.108275 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:07.108278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:07.108320 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:07.108437 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:07.108756 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.109731 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.111497 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.112253 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.112375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:07.112433 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:07.112729 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:15:07.112819 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260807 15:15:07.113255 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.107781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:40105 (local address 127.31.230.195:47001)
0807 15:15:07.108206 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:07.108219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:07.108367 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:07.108634 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:07.108637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:07.108653 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:07.108787 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:07.108793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.109896 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.109899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:07.111341 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:07.111352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.112351 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.112355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:07.112580 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:07.112604 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:15:07.112950 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 15:15:07.113044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":171,"thread_start_us":156,"threads_started":1}
I20260807 15:15:07.113564 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.107898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:47001 (local address 127.31.230.193:40105)
0807 15:15:07.108061 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:07.108067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:07.108325 (+ 258us) server_negotiation.cc:407] Connection header received
0807 15:15:07.108484 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:07.108488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:07.108537 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:07.108622 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:07.108928 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.109778 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.111497 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.112201 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.112375 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:07.112433 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:07.113311 (+ 878us) server_negotiation.cc:299] Negotiation successful
0807 15:15:07.113417 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:15:07.113802 6478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c85bd63cec84b78b365ea94072f5ab6" candidate_uuid: "35ad7d4796d04289b6aa08a3e2b42329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b3f7404e864d45a43c3c6e81019953" is_pre_election: true
I20260807 15:15:07.114184 6478 raft_consensus.cc:2454] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35ad7d4796d04289b6aa08a3e2b42329 in term 0.
I20260807 15:15:07.114732 6281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c85bd63cec84b78b365ea94072f5ab6" candidate_uuid: "35ad7d4796d04289b6aa08a3e2b42329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7643d1fd02e5464baae74e23994b6dd9" is_pre_election: true
I20260807 15:15:07.114787 6586 leader_election.cc:302] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35ad7d4796d04289b6aa08a3e2b42329, 97b3f7404e864d45a43c3c6e81019953; no voters:
I20260807 15:15:07.115054 6281 raft_consensus.cc:2454] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35ad7d4796d04289b6aa08a3e2b42329 in term 0.
I20260807 15:15:07.115095 6775 raft_consensus.cc:2781] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:07.115196 6775 raft_consensus.cc:482] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:07.115288 6775 raft_consensus.cc:3037] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:07.116533 6775 raft_consensus.cc:504] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:07.116942 6775 leader_election.cc:288] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [CANDIDATE]: Term 1 election: Requested vote from peers 97b3f7404e864d45a43c3c6e81019953 (127.31.230.194:41413), 7643d1fd02e5464baae74e23994b6dd9 (127.31.230.193:40105)
I20260807 15:15:07.117367 6281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c85bd63cec84b78b365ea94072f5ab6" candidate_uuid: "35ad7d4796d04289b6aa08a3e2b42329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7643d1fd02e5464baae74e23994b6dd9"
I20260807 15:15:07.117376 6478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c85bd63cec84b78b365ea94072f5ab6" candidate_uuid: "35ad7d4796d04289b6aa08a3e2b42329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b3f7404e864d45a43c3c6e81019953"
I20260807 15:15:07.117532 6281 raft_consensus.cc:3037] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:07.117538 6478 raft_consensus.cc:3037] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:07.118700 6478 raft_consensus.cc:2454] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35ad7d4796d04289b6aa08a3e2b42329 in term 1.
I20260807 15:15:07.118818 6281 raft_consensus.cc:2454] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35ad7d4796d04289b6aa08a3e2b42329 in term 1.
I20260807 15:15:07.119102 6586 leader_election.cc:302] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35ad7d4796d04289b6aa08a3e2b42329, 97b3f7404e864d45a43c3c6e81019953; no voters:
I20260807 15:15:07.119360 6775 raft_consensus.cc:2781] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:07.119567 6775 raft_consensus.cc:686] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 1 LEADER]: Becoming Leader. State: Replica: 35ad7d4796d04289b6aa08a3e2b42329, State: Running, Role: LEADER
I20260807 15:15:07.119908 6775 consensus_queue.cc:237] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } }
I20260807 15:15:07.123404 6083 catalog_manager.cc:5495] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35ad7d4796d04289b6aa08a3e2b42329 (127.31.230.195). New cstate: current_term: 1 leader_uuid: "35ad7d4796d04289b6aa08a3e2b42329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35ad7d4796d04289b6aa08a3e2b42329" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38445 } health_report { overall_health: HEALTHY } } }
W20260807 15:15:07.156637 6742 tablet.cc:2369] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:07.159682 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.155943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.31.230.195:38445)
0807 15:15:07.156101 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:07.156108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:07.156127 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:15:07.156224 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:07.156226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:07.156287 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:07.156402 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:07.156683 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.157271 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.158195 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:07.158429 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:07.158542 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:07.158889 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:07.159290 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:07.159430 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:07.159471 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 15:15:07.159534 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 15:15:07.161043 6553 tablet.cc:2369] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:07.165997 6281 raft_consensus.cc:1261] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 1 FOLLOWER]: Refusing update from remote peer 35ad7d4796d04289b6aa08a3e2b42329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:07.165997 6478 raft_consensus.cc:1261] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 1 FOLLOWER]: Refusing update from remote peer 35ad7d4796d04289b6aa08a3e2b42329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:07.166829 6775 consensus_queue.cc:1035] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7643d1fd02e5464baae74e23994b6dd9" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:07.167035 6779 consensus_queue.cc:1035] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b3f7404e864d45a43c3c6e81019953" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:07.174234 6789 mvcc.cc:205] Tried to move back new op lower bound from 7315929936543653888 to 7315929936366211072. Current Snapshot: MvccSnapshot[applied={T|T < 7315929936543653888}]
I20260807 15:15:07.174805 6784 mvcc.cc:205] Tried to move back new op lower bound from 7315929936543653888 to 7315929936366211072. Current Snapshot: MvccSnapshot[applied={T|T < 7315929936543653888}]
I20260807 15:15:07.183708 6083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:07.183879 6083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:07.186385 6083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3919d7297d384db5937e5b5fb9d859e4: Sending DeleteTablet for 3 replicas of tablet 3c85bd63cec84b78b365ea94072f5ab6
I20260807 15:15:07.187175 6450 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c85bd63cec84b78b365ea94072f5ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:07 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58214
I20260807 15:15:07.187269 6254 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c85bd63cec84b78b365ea94072f5ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:07 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:45804
I20260807 15:15:07.187320 6651 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c85bd63cec84b78b365ea94072f5ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:07 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50330
I20260807 15:15:07.187942 6800 tablet_replica.cc:330] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: stopping tablet replica
I20260807 15:15:07.188050 6801 tablet_replica.cc:330] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9: stopping tablet replica
I20260807 15:15:07.188284 6800 raft_consensus.cc:2229] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:07.188410 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 6170
I20260807 15:15:07.188563 6801 raft_consensus.cc:2229] T 3c85bd63cec84b78b365ea94072f5ab6 P 7643d1fd02e5464baae74e23994b6dd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:07.188674 6800 raft_consensus.cc:2258] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:07.190264 6800 ts_tablet_manager.cc:1905] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:07.194027 6800 ts_tablet_manager.cc:1918] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:07.194152 6800 log.cc:1196] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/3c85bd63cec84b78b365ea94072f5ab6
I20260807 15:15:07.194492 6800 ts_tablet_manager.cc:1939] T 3c85bd63cec84b78b365ea94072f5ab6 P 35ad7d4796d04289b6aa08a3e2b42329: Deleting consensus metadata
I20260807 15:15:07.195647 6063 catalog_manager.cc:4841] TS 35ad7d4796d04289b6aa08a3e2b42329 (127.31.230.195:38445): tablet 3c85bd63cec84b78b365ea94072f5ab6 (table test-table [id=b49ba9f9054c4f5fa56f14ebcc6d0ab9]) successfully deleted
I20260807 15:15:07.197651 6799 tablet_replica.cc:330] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: stopping tablet replica
I20260807 15:15:07.198012 6799 raft_consensus.cc:2229] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:07.198361 6799 raft_consensus.cc:2258] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:07.199723 6799 ts_tablet_manager.cc:1905] T 3c85bd63cec84b78b365ea94072f5ab6 P 97b3f7404e864d45a43c3c6e81019953: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:15:07.200619 6062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:40105 (error 108)
I20260807 15:15:07.201579 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 6356
I20260807 15:15:07.201848 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.201451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:40105 (local address 127.0.0.1:45812)
0807 15:15:07.201659 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:07.201737 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 15:15:07.202179 6062 catalog_manager.cc:4584] TS 7643d1fd02e5464baae74e23994b6dd9 (127.31.230.193:40105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c85bd63cec84b78b365ea94072f5ab6: Network error: Client connection negotiation failed: client connection to 127.31.230.193:40105: connect: Connection refused (error 111)
I20260807 15:15:07.211954 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 6558
I20260807 15:15:07.212231 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.211850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41413 (local address 127.0.0.1:58230)
0807 15:15:07.212013 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:07.212100 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.194:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 15:15:07.226531 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 6033
2026-08-07T15:15:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2675 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T15:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:07Z Disabled control of system clock
I20260807 15:15:07.265636 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45321
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:45321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:15:07.384239 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:07.384542 6815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:07.384581 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:07.388581 6815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:07.388640 6815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:07.388656 6815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:07.388674 6815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:07.388689 6815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:07.393411 6815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:45321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:07.394673 6815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:07.396353 6815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:07.405946 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:07.405997 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:07.406013 6815 server_base.cc:1034] running on GCE node
W20260807 15:15:07.406163 6820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:07.406951 6815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:07.408152 6815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:07.409353 6815 hybrid_clock.cc:648] HybridClock initialized: now 1786115707409330 us; error 33 us; skew 500 ppm
I20260807 15:15:07.415398 6815 webserver.cc:460] Webserver started at http://127.31.230.254:40851/ using document root <none> and password file <none>
I20260807 15:15:07.416025 6815 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:07.416093 6815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:07.416302 6815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:07.418728 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "1b384dddd4144108bf17963a855426b9"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "b41e3b237ce42103e07e3bdae0a9d321"
server_key_iv: "971ea99f1074df7e1a677ea33b355106"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.419297 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "1b384dddd4144108bf17963a855426b9"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "b41e3b237ce42103e07e3bdae0a9d321"
server_key_iv: "971ea99f1074df7e1a677ea33b355106"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.423446 6815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:07.426206 6833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.427663 6815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:07.427783 6815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "1b384dddd4144108bf17963a855426b9"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "b41e3b237ce42103e07e3bdae0a9d321"
server_key_iv: "971ea99f1074df7e1a677ea33b355106"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.427889 6815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:07.433166 6815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:07.433905 6815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:07.434085 6815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:07.441392 6815 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:45321
I20260807 15:15:07.441394 6907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:45321 every 8 connection(s)
I20260807 15:15:07.442545 6815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:07.446162 6908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:07.451864 6908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: Bootstrap starting.
I20260807 15:15:07.452682 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 6815
I20260807 15:15:07.452809 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:07.453119 32667 external_mini_cluster.cc:1428] Setting key 9e34110956ce0b29ca5411f0ca83f90b
I20260807 15:15:07.455056 6908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:07.455842 6908 log.cc:824] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:07.458314 6908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: No bootstrap required, opened a new log
I20260807 15:15:07.459343 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.454486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.31.230.254:45321)
0807 15:15:07.455318 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:07.455345 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:07.455390 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:15:07.456205 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:07.456237 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:07.456630 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:07.457114 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:07.457969 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:07.457973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:07.457985 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:07.457995 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:07.458051 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:07.458075 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:07.458197 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:07.458319 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:15:07.458597 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":172,"threads_started":1}
I20260807 15:15:07.461966 6908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b384dddd4144108bf17963a855426b9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45321 } }
I20260807 15:15:07.462218 6908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:07.462314 6908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b384dddd4144108bf17963a855426b9, State: Initialized, Role: FOLLOWER
I20260807 15:15:07.462759 6908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b384dddd4144108bf17963a855426b9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45321 } }
I20260807 15:15:07.462918 6908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:07.463008 6908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:07.463128 6908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:07.464097 6908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b384dddd4144108bf17963a855426b9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45321 } }
I20260807 15:15:07.464462 6908 leader_election.cc:302] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b384dddd4144108bf17963a855426b9; no voters:
I20260807 15:15:07.464789 6908 leader_election.cc:288] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:07.464920 6915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:07.465219 6915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1b384dddd4144108bf17963a855426b9, State: Running, Role: LEADER
I20260807 15:15:07.465544 6915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b384dddd4144108bf17963a855426b9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45321 } }
I20260807 15:15:07.466143 6908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:07.467330 6917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b384dddd4144108bf17963a855426b9. Latest consensus state: current_term: 1 leader_uuid: "1b384dddd4144108bf17963a855426b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b384dddd4144108bf17963a855426b9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45321 } } }
I20260807 15:15:07.467464 6917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:07.467310 6916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b384dddd4144108bf17963a855426b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b384dddd4144108bf17963a855426b9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45321 } } }
I20260807 15:15:07.467724 6916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9 [sys.catalog]: This master's current role is: LEADER
W20260807 15:15:07.471280 6934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:07.471390 6934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:07.471509 6925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:07.472244 6925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:07.477699 6925 catalog_manager.cc:1347] Generated new cluster ID: 897b80cb23434605b8c4f2e5a088b3d6
I20260807 15:15:07.477792 6925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:07.511134 6925 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:07.511508 6925 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:07.519694 6925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: Generated new TSK 0
I20260807 15:15:07.520499 6925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:07.529486 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45321
--builtin_ntp_servers=127.31.230.212:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:15:07.654347 6940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:07.654678 6940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:07.654750 6940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:07.658806 6940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:07.658901 6940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:07.658998 6940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:07.664068 6940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:07.665334 6940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:07.667019 6940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:07.677285 6948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:07.677235 6947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:07.677399 6951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:07.677481 6940 server_base.cc:1034] running on GCE node
I20260807 15:15:07.678015 6940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:07.678689 6940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:07.679890 6940 hybrid_clock.cc:648] HybridClock initialized: now 1786115707679861 us; error 55 us; skew 500 ppm
I20260807 15:15:07.681887 6940 webserver.cc:460] Webserver started at http://127.31.230.193:45807/ using document root <none> and password file <none>
I20260807 15:15:07.682498 6940 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:07.682600 6940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:07.682853 6940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:07.684984 6940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "fe56f57e4719c47ce9a52beb24691988"
server_key_iv: "25bd222fd4db316a6e3d6edc0f85f8ce"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.685642 6940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "fe56f57e4719c47ce9a52beb24691988"
server_key_iv: "25bd222fd4db316a6e3d6edc0f85f8ce"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.689378 6940 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:07.692144 6958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.693354 6940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:07.693503 6940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "fe56f57e4719c47ce9a52beb24691988"
server_key_iv: "25bd222fd4db316a6e3d6edc0f85f8ce"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.693674 6940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:07.698338 6940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:07.698977 6940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:07.699168 6940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:07.699718 6940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:07.700644 6940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:07.700716 6940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.700788 6940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:07.700839 6940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.709702 6940 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:33077
I20260807 15:15:07.709738 7118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:33077 every 8 connection(s)
I20260807 15:15:07.711010 6940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:07.717700 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.713074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:36649 (local address 127.31.230.254:45321)
0807 15:15:07.713221 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:07.713225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:07.714448 (+ 1223us) server_negotiation.cc:407] Connection header received
0807 15:15:07.715903 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:07.715907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:07.715953 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:07.716036 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:07.717174 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:07.717178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:07.717181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:07.717202 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:07.717223 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:07.717233 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:07.717339 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:07.717490 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:15:07.717559 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:15:07.718217 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 6940
I20260807 15:15:07.718382 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:07.718650 7127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.713576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45321 (local address 127.31.230.193:36649)
0807 15:15:07.714294 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:07.714336 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:07.715676 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:07.716201 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:07.716207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:07.716306 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:07.716905 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:07.716907 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:07.716950 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:07.717018 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:07.717321 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:07.717330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:07.717336 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:15:07.717529 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 15:15:07.717803 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":132,"threads_started":1}
I20260807 15:15:07.718691 32667 external_mini_cluster.cc:1428] Setting key d47cdf546d33ee56c38f01c10e4333a2
I20260807 15:15:07.720268 7120 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45321
I20260807 15:15:07.720536 7120 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:07.720896 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45321
--builtin_ntp_servers=127.31.230.212:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 15:15:07.721020 7120 heartbeater.cc:511] Master 127.31.230.254:45321 requested a full tablet report, sending...
I20260807 15:15:07.722769 6860 ts_manager.cc:194] Registered new tserver with Master: 10aaa50a286a42f5a7c4e99d0ed6a0e7 (127.31.230.193:33077)
W20260807 15:15:07.840181 7128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:07.840436 7128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:07.840476 7128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:07.844540 7128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:07.844599 7128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:07.844653 7128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:07.849370 7128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:07.850428 7128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:07.851769 7128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:07.861299 7133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:07.861467 7128 server_base.cc:1034] running on GCE node
W20260807 15:15:07.861306 7134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:07.861653 7136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:07.862051 7128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:07.862695 7128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:07.863898 7128 hybrid_clock.cc:648] HybridClock initialized: now 1786115707863863 us; error 46 us; skew 500 ppm
I20260807 15:15:07.865839 7128 webserver.cc:460] Webserver started at http://127.31.230.194:40085/ using document root <none> and password file <none>
I20260807 15:15:07.866465 7128 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:07.866581 7128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:07.866825 7128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:07.869107 7128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "de3c23cdf95a43c59baea79e50541663"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "ffb511235b9dfacd6143b21223959625"
server_key_iv: "aeb81167ac053653c92274f888fb85f4"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.869745 7128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "de3c23cdf95a43c59baea79e50541663"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "ffb511235b9dfacd6143b21223959625"
server_key_iv: "aeb81167ac053653c92274f888fb85f4"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.873660 7128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:15:07.876394 7142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.877696 7128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:07.877827 7128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "de3c23cdf95a43c59baea79e50541663"
format_stamp: "Formatted at 2026-08-07 15:15:07 on dist-test-slave-gkw7"
server_key: "ffb511235b9dfacd6143b21223959625"
server_key_iv: "aeb81167ac053653c92274f888fb85f4"
server_key_version: "encryptionkey@0"
I20260807 15:15:07.877935 7128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:07.882587 7128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:07.883198 7128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:07.883389 7128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:07.883953 7128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:07.884861 7128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:07.884913 7128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.884980 7128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:07.885017 7128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:07.894145 7128 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:41661
I20260807 15:15:07.894164 7305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:41661 every 8 connection(s)
I20260807 15:15:07.895318 7128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:07.899147 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 7128
I20260807 15:15:07.899289 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:07.899626 32667 external_mini_cluster.cc:1428] Setting key d59f3b0971b7d0e74b69983809bfbc0f
I20260807 15:15:07.901727 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45321
--builtin_ntp_servers=127.31.230.212:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 15:15:07.903378 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.898412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:36327 (local address 127.31.230.254:45321)
0807 15:15:07.898571 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:07.898575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:07.899989 (+ 1414us) server_negotiation.cc:407] Connection header received
0807 15:15:07.901499 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:07.901503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:07.901570 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:07.901650 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:07.902889 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:07.902893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:07.902896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:07.902908 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:07.902931 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:07.902940 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:07.903043 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:07.903189 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 15:15:07.903234 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:15:07.904299 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:07.899031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45321 (local address 127.31.230.194:36327)
0807 15:15:07.899835 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:07.899877 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:07.901297 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:07.901872 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:07.901884 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:07.901999 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:07.902622 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:07.902623 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:07.902664 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:07.902732 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:07.903031 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:07.903041 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:07.903048 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:15:07.903234 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 15:15:07.903478 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":151,"threads_started":1}
I20260807 15:15:07.905493 7306 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45321
I20260807 15:15:07.905752 7306 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:07.906167 7306 heartbeater.cc:511] Master 127.31.230.254:45321 requested a full tablet report, sending...
I20260807 15:15:07.907276 6860 ts_manager.cc:194] Registered new tserver with Master: de3c23cdf95a43c59baea79e50541663 (127.31.230.194:41661)
W20260807 15:15:08.019243 7312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:08.019630 7312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:08.019696 7312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:08.023969 7312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:08.024029 7312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:08.024092 7312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:08.028883 7312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:08.030054 7312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:08.031540 7312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:08.041858 7322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:08.041858 7320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:08.041862 7318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:08.042200 7312 server_base.cc:1034] running on GCE node
I20260807 15:15:08.042618 7312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:08.043280 7312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:08.044500 7312 hybrid_clock.cc:648] HybridClock initialized: now 1786115708044473 us; error 36 us; skew 500 ppm
I20260807 15:15:08.046726 7312 webserver.cc:460] Webserver started at http://127.31.230.195:37009/ using document root <none> and password file <none>
I20260807 15:15:08.047405 7312 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:08.047475 7312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:08.047686 7312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:08.049845 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "c18ca9216a4c40a292ef51bb16539f15"
format_stamp: "Formatted at 2026-08-07 15:15:08 on dist-test-slave-gkw7"
server_key: "f9eb9f5342471ef9fbbb012a2eafb511"
server_key_iv: "35989428bd223459c3642b9c0f4c8a9c"
server_key_version: "encryptionkey@0"
I20260807 15:15:08.050362 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "c18ca9216a4c40a292ef51bb16539f15"
format_stamp: "Formatted at 2026-08-07 15:15:08 on dist-test-slave-gkw7"
server_key: "f9eb9f5342471ef9fbbb012a2eafb511"
server_key_iv: "35989428bd223459c3642b9c0f4c8a9c"
server_key_version: "encryptionkey@0"
I20260807 15:15:08.054083 7312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 15:15:08.056787 7328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:08.058444 7312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:08.058573 7312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "c18ca9216a4c40a292ef51bb16539f15"
format_stamp: "Formatted at 2026-08-07 15:15:08 on dist-test-slave-gkw7"
server_key: "f9eb9f5342471ef9fbbb012a2eafb511"
server_key_iv: "35989428bd223459c3642b9c0f4c8a9c"
server_key_version: "encryptionkey@0"
I20260807 15:15:08.058681 7312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:08.063791 7312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:08.064429 7312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:08.064585 7312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:08.065136 7312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:08.066239 7312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:08.066295 7312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:08.066339 7312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:08.066355 7312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:08.076472 7312 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:37495
I20260807 15:15:08.076496 7487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:37495 every 8 connection(s)
I20260807 15:15:08.077515 7312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:08.079447 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 7312
I20260807 15:15:08.079595 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:08.079936 32667 external_mini_cluster.cc:1428] Setting key d3c1b579686d34d3d1912b0004859f3b
I20260807 15:15:08.084824 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.079665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:47397 (local address 127.31.230.254:45321)
0807 15:15:08.079838 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.079844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.081253 (+ 1409us) server_negotiation.cc:407] Connection header received
0807 15:15:08.083029 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.083032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.083076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.083151 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.084321 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.084325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.084328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.084341 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.084367 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.084381 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.084471 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.084633 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.084689 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:15:08.085788 7491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.080184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45321 (local address 127.31.230.195:47397)
0807 15:15:08.081109 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.081154 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.082822 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.083327 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.083334 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.083421 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:08.084057 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:08.084059 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:08.084100 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:08.084168 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:08.084472 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:08.084483 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:08.084491 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:15:08.084680 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 15:15:08.084937 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":177,"threads_started":1}
I20260807 15:15:08.086850 7488 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45321
I20260807 15:15:08.087067 7488 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:08.087477 7488 heartbeater.cc:511] Master 127.31.230.254:45321 requested a full tablet report, sending...
I20260807 15:15:08.088457 6860 ts_manager.cc:194] Registered new tserver with Master: c18ca9216a4c40a292ef51bb16539f15 (127.31.230.195:37495)
I20260807 15:15:08.094470 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:08.097985 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.096439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.31.230.254:45321)
0807 15:15:08.096621 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.096628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.096755 (+ 127us) server_negotiation.cc:407] Connection header received
0807 15:15:08.096855 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.096857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.096888 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.096951 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.097613 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.097616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.097618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.097629 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.097648 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.097655 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.097742 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.097791 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.097823 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 15:15:08.100921 6860 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:08.103992 6860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:08.116755 7509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.112734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:33077 (local address 127.0.0.1:58048)
0807 15:15:08.113710 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.113767 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.113871 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.114733 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.114737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.114809 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:08.115488 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:08.115490 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:08.115511 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:08.115605 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:08.116468 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:08.116472 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:08.116476 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:08.116537 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:15:08.116592 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":94,"threads_started":1}
I20260807 15:15:08.116813 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.113644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54330 (local address 127.31.230.194:41661)
0807 15:15:08.114354 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.114364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.114382 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:15:08.114426 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.114430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.114565 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.114729 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.115994 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.116001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.116032 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.116049 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.116102 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.116141 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.116264 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.116568 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.116648 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":122,"threads_started":1}
I20260807 15:15:08.116878 7508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.113000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.31.230.195:37495)
0807 15:15:08.114214 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.114229 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.114259 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:15:08.114323 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.114331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.114554 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.114730 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.116094 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.116099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.116107 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.116119 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.116157 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.116204 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.116303 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.116668 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.116729 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":145,"threads_started":1}
I20260807 15:15:08.117074 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.112734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37495 (local address 127.0.0.1:58206)
0807 15:15:08.114064 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.114077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.114148 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.115502 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.115505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.115521 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:08.115916 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:08.115918 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:08.115929 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:08.115956 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:08.116341 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:08.116344 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:08.116347 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:08.116443 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:15:08.116531 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":676,"thread_start_us":124,"threads_started":1}
I20260807 15:15:08.117385 7504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.113156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41661 (local address 127.0.0.1:54330)
0807 15:15:08.113700 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.113765 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.113876 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.114718 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.114725 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.114809 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:08.115594 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:08.115596 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:08.115605 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:08.115630 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:08.116312 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:08.116324 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:08.116332 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:15:08.116443 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:15:08.117035 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":502,"thread_start_us":123,"threads_started":1}
I20260807 15:15:08.117798 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.112894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.31.230.193:33077)
0807 15:15:08.114310 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.114321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.114341 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:15:08.114390 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.114395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.114562 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.115140 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.116227 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.116234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.116246 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.116258 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.116308 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.116340 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.117402 (+ 1062us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.117498 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.117630 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":150,"threads_started":1}
I20260807 15:15:08.120512 7019 tablet_service.cc:1467] Processing CreateTablet for tablet ec3775af4edd4c00a1e01fc0e86e1dd6 (DEFAULT_TABLE table=test-table [id=3501ab6a5057477387044762184e5097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:08.120512 7205 tablet_service.cc:1467] Processing CreateTablet for tablet ec3775af4edd4c00a1e01fc0e86e1dd6 (DEFAULT_TABLE table=test-table [id=3501ab6a5057477387044762184e5097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:08.120515 7400 tablet_service.cc:1467] Processing CreateTablet for tablet ec3775af4edd4c00a1e01fc0e86e1dd6 (DEFAULT_TABLE table=test-table [id=3501ab6a5057477387044762184e5097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:08.121644 7019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec3775af4edd4c00a1e01fc0e86e1dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:08.121651 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec3775af4edd4c00a1e01fc0e86e1dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:08.121651 7205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec3775af4edd4c00a1e01fc0e86e1dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:08.127923 7513 tablet_bootstrap.cc:492] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: Bootstrap starting.
I20260807 15:15:08.129930 7515 tablet_bootstrap.cc:492] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Bootstrap starting.
I20260807 15:15:08.131539 7513 tablet_bootstrap.cc:654] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:08.129359 7514 tablet_bootstrap.cc:492] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Bootstrap starting.
I20260807 15:15:08.132120 7515 tablet_bootstrap.cc:654] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:08.132469 7513 log.cc:824] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:08.132943 7515 log.cc:824] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:08.134014 7514 tablet_bootstrap.cc:654] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:08.134709 7513 tablet_bootstrap.cc:492] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: No bootstrap required, opened a new log
I20260807 15:15:08.134737 7514 log.cc:824] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:08.134914 7513 ts_tablet_manager.cc:1397] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:15:08.134949 7515 tablet_bootstrap.cc:492] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: No bootstrap required, opened a new log
I20260807 15:15:08.135109 7515 ts_tablet_manager.cc:1397] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:15:08.136857 7514 tablet_bootstrap.cc:492] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: No bootstrap required, opened a new log
I20260807 15:15:08.137081 7514 ts_tablet_manager.cc:1397] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 15:15:08.139642 7515 raft_consensus.cc:348] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.139642 7513 raft_consensus.cc:348] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.139928 7513 raft_consensus.cc:374] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:08.139928 7515 raft_consensus.cc:374] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:08.140005 7515 raft_consensus.cc:729] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18ca9216a4c40a292ef51bb16539f15, State: Initialized, Role: FOLLOWER
I20260807 15:15:08.140005 7513 raft_consensus.cc:729] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10aaa50a286a42f5a7c4e99d0ed6a0e7, State: Initialized, Role: FOLLOWER
I20260807 15:15:08.140488 7513 consensus_queue.cc:260] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.140488 7515 consensus_queue.cc:260] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.141234 7120 heartbeater.cc:503] Master 127.31.230.254:45321 was elected leader, sending a full tablet report...
I20260807 15:15:08.141050 7514 raft_consensus.cc:348] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.141323 7514 raft_consensus.cc:374] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:08.141418 7514 raft_consensus.cc:729] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3c23cdf95a43c59baea79e50541663, State: Initialized, Role: FOLLOWER
I20260807 15:15:08.141688 7488 heartbeater.cc:503] Master 127.31.230.254:45321 was elected leader, sending a full tablet report...
I20260807 15:15:08.141947 7514 consensus_queue.cc:260] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.141559 7513 ts_tablet_manager.cc:1428] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:15:08.142997 7514 ts_tablet_manager.cc:1428] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:15:08.143110 7306 heartbeater.cc:503] Master 127.31.230.254:45321 was elected leader, sending a full tablet report...
I20260807 15:15:08.144944 7515 ts_tablet_manager.cc:1428] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260807 15:15:08.146401 7308 tablet.cc:2369] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:08.212323 7121 tablet.cc:2369] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:08.300277 7522 raft_consensus.cc:482] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:08.300515 7522 raft_consensus.cc:504] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.301753 7522 leader_election.cc:288] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c18ca9216a4c40a292ef51bb16539f15 (127.31.230.195:37495), de3c23cdf95a43c59baea79e50541663 (127.31.230.194:41661)
I20260807 15:15:08.304088 7508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.302300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:60171 (local address 127.31.230.195:37495)
0807 15:15:08.302461 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.302466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.302550 (+ 84us) server_negotiation.cc:407] Connection header received
0807 15:15:08.302705 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.302709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.302754 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.302853 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.303669 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.303672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.303674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.303686 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.303707 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.303718 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.303836 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.303914 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.303953 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:15:08.304105 7526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.302054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37495 (local address 127.31.230.193:60171)
0807 15:15:08.302445 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.302462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.302584 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.302897 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.302900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.302915 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:08.303510 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:08.303512 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:08.303521 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:08.303546 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:08.303822 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:08.303825 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:08.303829 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:08.303910 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:15:08.303953 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":101,"threads_started":1}
I20260807 15:15:08.304266 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.302347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:34119 (local address 127.31.230.194:41661)
0807 15:15:08.302472 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.302476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.302711 (+ 235us) server_negotiation.cc:407] Connection header received
0807 15:15:08.302881 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.302884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.302924 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.303003 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.303866 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.303870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.303872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.303885 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.303905 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.303917 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.304020 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.304100 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.304139 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:15:08.304293 7527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.302054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41661 (local address 127.31.230.193:34119)
0807 15:15:08.302624 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.302639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.302776 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.303052 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.303055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.303070 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:08.303700 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:08.303702 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:08.303711 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:08.303736 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:08.304016 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:08.304019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:08.304022 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:08.304099 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:15:08.304139 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":16,"thread_start_us":87,"threads_started":1}
I20260807 15:15:08.304790 7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3775af4edd4c00a1e01fc0e86e1dd6" candidate_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18ca9216a4c40a292ef51bb16539f15" is_pre_election: true
I20260807 15:15:08.304888 7236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3775af4edd4c00a1e01fc0e86e1dd6" candidate_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de3c23cdf95a43c59baea79e50541663" is_pre_election: true
I20260807 15:15:08.305140 7236 raft_consensus.cc:2454] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10aaa50a286a42f5a7c4e99d0ed6a0e7 in term 0.
I20260807 15:15:08.305140 7424 raft_consensus.cc:2454] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10aaa50a286a42f5a7c4e99d0ed6a0e7 in term 0.
I20260807 15:15:08.305660 6966 leader_election.cc:302] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10aaa50a286a42f5a7c4e99d0ed6a0e7, c18ca9216a4c40a292ef51bb16539f15, de3c23cdf95a43c59baea79e50541663; no voters:
I20260807 15:15:08.305977 7522 raft_consensus.cc:2781] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:08.306064 7522 raft_consensus.cc:482] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:08.306116 7522 raft_consensus.cc:3037] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:08.307193 7522 raft_consensus.cc:504] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.307545 7522 leader_election.cc:288] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [CANDIDATE]: Term 1 election: Requested vote from peers c18ca9216a4c40a292ef51bb16539f15 (127.31.230.195:37495), de3c23cdf95a43c59baea79e50541663 (127.31.230.194:41661)
I20260807 15:15:08.307898 7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3775af4edd4c00a1e01fc0e86e1dd6" candidate_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18ca9216a4c40a292ef51bb16539f15"
I20260807 15:15:08.307915 7236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3775af4edd4c00a1e01fc0e86e1dd6" candidate_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de3c23cdf95a43c59baea79e50541663"
I20260807 15:15:08.308049 7236 raft_consensus.cc:3037] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:08.307998 7424 raft_consensus.cc:3037] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:08.309083 7236 raft_consensus.cc:2454] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10aaa50a286a42f5a7c4e99d0ed6a0e7 in term 1.
I20260807 15:15:08.309175 7424 raft_consensus.cc:2454] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10aaa50a286a42f5a7c4e99d0ed6a0e7 in term 1.
I20260807 15:15:08.309420 6961 leader_election.cc:302] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10aaa50a286a42f5a7c4e99d0ed6a0e7, de3c23cdf95a43c59baea79e50541663; no voters:
I20260807 15:15:08.309639 7522 raft_consensus.cc:2781] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:08.309899 7522 raft_consensus.cc:686] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 1 LEADER]: Becoming Leader. State: Replica: 10aaa50a286a42f5a7c4e99d0ed6a0e7, State: Running, Role: LEADER
I20260807 15:15:08.310259 7522 consensus_queue.cc:237] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } }
I20260807 15:15:08.313413 6860 catalog_manager.cc:5495] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10aaa50a286a42f5a7c4e99d0ed6a0e7 (127.31.230.193). New cstate: current_term: 1 leader_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10aaa50a286a42f5a7c4e99d0ed6a0e7" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 33077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 } health_report { overall_health: UNKNOWN } } }
W20260807 15:15:08.328661 7490 tablet.cc:2369] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:08.355319 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.353834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.31.230.193:33077)
0807 15:15:08.353969 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.353973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.353986 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:08.354098 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.354100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.354165 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.354245 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.354931 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.354934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.354939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.354983 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.355008 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.355022 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.355123 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.355177 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.355213 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:15:08.361481 7236 raft_consensus.cc:1261] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 1 FOLLOWER]: Refusing update from remote peer 10aaa50a286a42f5a7c4e99d0ed6a0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:08.361709 7424 raft_consensus.cc:1261] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 1 FOLLOWER]: Refusing update from remote peer 10aaa50a286a42f5a7c4e99d0ed6a0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:08.362491 7528 consensus_queue.cc:1035] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de3c23cdf95a43c59baea79e50541663" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:08.362758 7522 consensus_queue.cc:1035] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c18ca9216a4c40a292ef51bb16539f15" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:08.372551 7532 mvcc.cc:205] Tried to move back new op lower bound from 7315929941441167360 to 7315929941241307136. Current Snapshot: MvccSnapshot[applied={T|T < 7315929941441167360}]
I20260807 15:15:08.373335 7531 mvcc.cc:205] Tried to move back new op lower bound from 7315929941441167360 to 7315929941241307136. Current Snapshot: MvccSnapshot[applied={T|T < 7315929941441167360}]
I20260807 15:15:08.382495 6858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:08.382688 6858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:08.385511 6858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b384dddd4144108bf17963a855426b9: Sending DeleteTablet for 3 replicas of tablet ec3775af4edd4c00a1e01fc0e86e1dd6
I20260807 15:15:08.386479 7019 tablet_service.cc:1514] Processing DeleteTablet for tablet ec3775af4edd4c00a1e01fc0e86e1dd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:08 UTC) from {username='slave'} at 127.0.0.1:58048
I20260807 15:15:08.386505 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet ec3775af4edd4c00a1e01fc0e86e1dd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:08 UTC) from {username='slave'} at 127.0.0.1:58206
I20260807 15:15:08.386837 7205 tablet_service.cc:1514] Processing DeleteTablet for tablet ec3775af4edd4c00a1e01fc0e86e1dd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:08 UTC) from {username='slave'} at 127.0.0.1:54330
I20260807 15:15:08.387064 7546 tablet_replica.cc:330] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: stopping tablet replica
I20260807 15:15:08.387180 7545 tablet_replica.cc:330] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7: stopping tablet replica
I20260807 15:15:08.387398 7546 raft_consensus.cc:2229] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:08.387631 7545 raft_consensus.cc:2229] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:08.387718 7546 raft_consensus.cc:2258] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:08.388173 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 6940
I20260807 15:15:08.388199 7545 raft_consensus.cc:2258] T ec3775af4edd4c00a1e01fc0e86e1dd6 P 10aaa50a286a42f5a7c4e99d0ed6a0e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:08.389868 7546 ts_tablet_manager.cc:1905] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:08.392122 7546 ts_tablet_manager.cc:1918] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:08.392246 7546 log.cc:1196] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/ec3775af4edd4c00a1e01fc0e86e1dd6
I20260807 15:15:08.392571 7546 ts_tablet_manager.cc:1939] T ec3775af4edd4c00a1e01fc0e86e1dd6 P c18ca9216a4c40a292ef51bb16539f15: Deleting consensus metadata
I20260807 15:15:08.394817 7547 tablet_replica.cc:330] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: stopping tablet replica
I20260807 15:15:08.395150 6837 catalog_manager.cc:4841] TS c18ca9216a4c40a292ef51bb16539f15 (127.31.230.195:37495): tablet ec3775af4edd4c00a1e01fc0e86e1dd6 (table test-table [id=3501ab6a5057477387044762184e5097]) successfully deleted
I20260807 15:15:08.395217 7547 raft_consensus.cc:2229] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:08.395586 7547 raft_consensus.cc:2258] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:08.397151 7547 ts_tablet_manager.cc:1905] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:08.399794 7547 ts_tablet_manager.cc:1918] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:08.399921 7547 log.cc:1196] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/ec3775af4edd4c00a1e01fc0e86e1dd6
I20260807 15:15:08.400281 7547 ts_tablet_manager.cc:1939] T ec3775af4edd4c00a1e01fc0e86e1dd6 P de3c23cdf95a43c59baea79e50541663: Deleting consensus metadata
W20260807 15:15:08.400899 6834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:33077 (error 108)
I20260807 15:15:08.401857 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 7128
I20260807 15:15:08.402192 7504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.401861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:33077 (local address 127.0.0.1:58072)
0807 15:15:08.402008 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.402091 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 15:15:08.415066 6834 catalog_manager.cc:4584] TS 10aaa50a286a42f5a7c4e99d0ed6a0e7 (127.31.230.193:33077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec3775af4edd4c00a1e01fc0e86e1dd6: Network error: Client connection negotiation failed: client connection to 127.31.230.193:33077: connect: Connection refused (error 111)
I20260807 15:15:08.426646 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 7312
I20260807 15:15:08.434151 7504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.426922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41661 (local address 127.0.0.1:54338)
0807 15:15:08.433919 (+ 6997us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.434021 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.194:41661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6837}
I20260807 15:15:08.449784 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 6815
2026-08-07T15:15:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1224 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 15:15:08.481069 32667 mini_oidc.cc:152] Starting JWKS server
I20260807 15:15:08.483232 32667 webserver.cc:460] Webserver started at https://127.0.0.1:38341/ using document root <none> and password file <none>
I20260807 15:15:08.483408 32667 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:15:08.483706 32667 webserver.cc:460] Webserver started at http://127.0.0.1:34845/ using document root <none> and password file <none>
2026-08-07T15:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:08Z Disabled control of system clock
I20260807 15:15:08.495401 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:40791
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:40791
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38341/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/testchainca.cert with env {}
W20260807 15:15:08.611853 7557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:08.612129 7557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:08.612170 7557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:08.616098 7557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:08.616158 7557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:08.616173 7557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:08.616190 7557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:08.616206 7557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:15:08.616221 7557 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:15:08.616235 7557 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38341/jwks.json
I20260807 15:15:08.621824 7557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:40791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38341/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:08.622876 7557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:08.623854 7557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:08.632746 7569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:08.632709 7567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:08.632853 7557 server_base.cc:1034] running on GCE node
W20260807 15:15:08.632722 7566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:08.633669 7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:08.634899 7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:08.636096 7557 hybrid_clock.cc:648] HybridClock initialized: now 1786115708636085 us; error 29 us; skew 500 ppm
I20260807 15:15:08.638034 7557 webserver.cc:460] Webserver started at http://127.31.230.254:36107/ using document root <none> and password file <none>
I20260807 15:15:08.638649 7557 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:08.638747 7557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:08.638988 7557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:08.641215 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "6fe378fc737b4d4eae3225999a5471d3"
format_stamp: "Formatted at 2026-08-07 15:15:08 on dist-test-slave-gkw7"
server_key: "394cb3df9c7860f8e9933de3bafbcdcf"
server_key_iv: "666be55a6311b84c8edaddcdca4e16e8"
server_key_version: "encryptionkey@0"
I20260807 15:15:08.641919 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "6fe378fc737b4d4eae3225999a5471d3"
format_stamp: "Formatted at 2026-08-07 15:15:08 on dist-test-slave-gkw7"
server_key: "394cb3df9c7860f8e9933de3bafbcdcf"
server_key_iv: "666be55a6311b84c8edaddcdca4e16e8"
server_key_version: "encryptionkey@0"
I20260807 15:15:08.645651 7557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:15:08.648043 7581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:08.649474 7557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:08.649679 7557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "6fe378fc737b4d4eae3225999a5471d3"
format_stamp: "Formatted at 2026-08-07 15:15:08 on dist-test-slave-gkw7"
server_key: "394cb3df9c7860f8e9933de3bafbcdcf"
server_key_iv: "666be55a6311b84c8edaddcdca4e16e8"
server_key_version: "encryptionkey@0"
I20260807 15:15:08.649811 7557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:08.669258 7557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:08.670121 7557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:08.670467 7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:08.678099 7557 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:40791
I20260807 15:15:08.678203 7645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:40791 every 8 connection(s)
I20260807 15:15:08.679153 7557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:08.682336 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 7557
I20260807 15:15:08.682471 7646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:08.682488 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:08.682775 32667 external_mini_cluster.cc:1428] Setting key 136699f5b6524ad2c3b917c990d1e7e5
I20260807 15:15:08.689445 7646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3: Bootstrap starting.
I20260807 15:15:08.692132 7646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:08.692879 7646 log.cc:824] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:08.693091 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.684233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.31.230.254:40791)
0807 15:15:08.684894 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.684915 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.684952 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:15:08.685746 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.685765 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.686160 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.686444 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:08.686791 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:08.687787 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:08.689047 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:08.689860 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:08.689993 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:08.691015 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:08.691020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:08.691033 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:08.691489 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:08.691546 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:08.691567 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:08.691660 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:08.691820 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:15:08.692276 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":142,"threads_started":1}
I20260807 15:15:08.695281 7646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3: No bootstrap required, opened a new log
I20260807 15:15:08.696167 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:08.699183 7646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe378fc737b4d4eae3225999a5471d3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40791 } }
I20260807 15:15:08.699445 7646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:08.699506 7646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe378fc737b4d4eae3225999a5471d3, State: Initialized, Role: FOLLOWER
I20260807 15:15:08.699970 7646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe378fc737b4d4eae3225999a5471d3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40791 } }
I20260807 15:15:08.700114 7646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:08.700162 7646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:08.700229 7646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:08.702663 7646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe378fc737b4d4eae3225999a5471d3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40791 } }
I20260807 15:15:08.702960 7646 leader_election.cc:302] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fe378fc737b4d4eae3225999a5471d3; no voters:
I20260807 15:15:08.703225 7646 leader_election.cc:288] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:08.703361 7653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:08.703603 7653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [term 1 LEADER]: Becoming Leader. State: Replica: 6fe378fc737b4d4eae3225999a5471d3, State: Running, Role: LEADER
I20260807 15:15:08.703917 7653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe378fc737b4d4eae3225999a5471d3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40791 } }
I20260807 15:15:08.704545 7646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:08.705670 7657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fe378fc737b4d4eae3225999a5471d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe378fc737b4d4eae3225999a5471d3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40791 } } }
I20260807 15:15:08.705814 7657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:08.706115 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fe378fc737b4d4eae3225999a5471d3. Latest consensus state: current_term: 1 leader_uuid: "6fe378fc737b4d4eae3225999a5471d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe378fc737b4d4eae3225999a5471d3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40791 } } }
I20260807 15:15:08.706214 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:08.706501 7670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:08.709206 7670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:08.714491 7670 catalog_manager.cc:1347] Generated new cluster ID: 7f6beedd52944055b3e78c09adf75dfb
I20260807 15:15:08.714570 7670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:08.749907 7670 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:08.751240 7670 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:08.759002 7670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fe378fc737b4d4eae3225999a5471d3: Generated new TSK 0
I20260807 15:15:08.759905 7670 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:15:08.798185 7696 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 15:15:08.798362 7696 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:15:08.799014 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:08.780901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.31.230.254:40791)
0807 15:15:08.781047 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:08.781052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:08.781247 (+ 195us) server_negotiation.cc:407] Connection header received
0807 15:15:08.781456 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:08.781460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:08.781523 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:08.781671 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:15:08.781935 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:08.782778 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:08.783750 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:08.784128 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:08.784233 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:08.784611 (+ 378us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:15:08.798537 (+ 13926us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38341/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:15:08.798866 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36456: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38341/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 15:15:08.799117 7649 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36456: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38341/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 15:15:08.800398 7695 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:08.780753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40791 (local address 127.0.0.1:36456)
0807 15:15:08.781159 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:08.781173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:08.781290 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:08.781685 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:08.781687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:08.781695 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:15:08.781825 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:08.781830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:08.782910 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:08.782912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:08.783633 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:08.783642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:08.784271 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:08.784274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:08.784350 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:08.784399 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:15:08.798822 (+ 14423us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38341/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:15:08.799048 (+ 226us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.230.254:40791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38341/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":130,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.230.254:40791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38341/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 15:15:08.801288 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 7557
2026-08-07T15:15:08Z chronyd exiting
I20260807 15:15:08.847895 32667 test_util.cc:175] -----------------------------------------------
I20260807 15:15:08.847987 32667 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115680745020-32667-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (368 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 15:15:08.849154 32667 mini_oidc.cc:152] Starting JWKS server
I20260807 15:15:08.850088 32667 webserver.cc:460] Webserver started at https://127.0.0.1:46281/ using document root <none> and password file <none>
I20260807 15:15:08.850168 32667 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:15:08.850387 32667 webserver.cc:460] Webserver started at http://127.0.0.1:41321/ using document root <none> and password file <none>
2026-08-07T15:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:08Z Disabled control of system clock
I20260807 15:15:08.860941 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34017
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:34017
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46281/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/testchainca.cert with env {}
W20260807 15:15:08.978379 7707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:08.978681 7707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:08.978724 7707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:08.982785 7707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:08.982854 7707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:08.982870 7707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:08.982887 7707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:08.982903 7707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:15:08.982919 7707 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:15:08.982932 7707 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46281/jwks.json
I20260807 15:15:08.988768 7707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:34017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46281/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:08.989914 7707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:08.990970 7707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:09.000641 7716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:09.000739 7707 server_base.cc:1034] running on GCE node
W20260807 15:15:09.000634 7714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:09.000638 7713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:09.001730 7707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:09.002957 7707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:09.004155 7707 hybrid_clock.cc:648] HybridClock initialized: now 1786115709004121 us; error 49 us; skew 500 ppm
I20260807 15:15:09.006179 7707 webserver.cc:460] Webserver started at http://127.31.230.254:33081/ using document root <none> and password file <none>
I20260807 15:15:09.007094 7707 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:09.007208 7707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:09.007498 7707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:09.009748 7707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "f784e211d1f043d0bf78f39f7a837cd7"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "e482ff902047661f0883531c9814e5c3"
server_key_iv: "ea18a1d6267b3da870fc68262f57d56d"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.010315 7707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "f784e211d1f043d0bf78f39f7a837cd7"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "e482ff902047661f0883531c9814e5c3"
server_key_iv: "ea18a1d6267b3da870fc68262f57d56d"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.014211 7707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:09.016891 7723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:09.018406 7707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:09.018517 7707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "f784e211d1f043d0bf78f39f7a837cd7"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "e482ff902047661f0883531c9814e5c3"
server_key_iv: "ea18a1d6267b3da870fc68262f57d56d"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.018611 7707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:09.054077 7707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:09.054807 7707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:09.054970 7707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:09.062708 7707 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:34017
I20260807 15:15:09.062742 7798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:34017 every 8 connection(s)
I20260807 15:15:09.063745 7707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:09.067391 7799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:09.068259 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 7707
I20260807 15:15:09.068408 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:09.068768 32667 external_mini_cluster.cc:1428] Setting key cea8d5ba0a6d4c3522a97936b23ecfe9
I20260807 15:15:09.074309 7799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7: Bootstrap starting.
I20260807 15:15:09.076898 7799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:09.077876 7799 log.cc:824] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:09.078646 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:09.070351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.31.230.254:34017)
0807 15:15:09.071042 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:09.071057 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:09.071094 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:15:09.071762 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:09.071780 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:09.072178 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:09.072544 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:09.072979 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.074025 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.075019 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.075506 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.075685 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.076630 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:09.076635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:09.076648 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:09.077096 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:09.077155 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:09.077177 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:09.077310 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:09.077461 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:15:09.077756 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":158,"threads_started":1}
I20260807 15:15:09.080343 7799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7: No bootstrap required, opened a new log
I20260807 15:15:09.081717 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:09.084702 7799 raft_consensus.cc:348] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f784e211d1f043d0bf78f39f7a837cd7" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34017 } }
I20260807 15:15:09.084968 7799 raft_consensus.cc:374] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:09.085067 7799 raft_consensus.cc:729] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f784e211d1f043d0bf78f39f7a837cd7, State: Initialized, Role: FOLLOWER
I20260807 15:15:09.085652 7799 consensus_queue.cc:260] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f784e211d1f043d0bf78f39f7a837cd7" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34017 } }
I20260807 15:15:09.085826 7799 raft_consensus.cc:388] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:09.085903 7799 raft_consensus.cc:482] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:09.086011 7799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:09.087057 7799 raft_consensus.cc:504] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f784e211d1f043d0bf78f39f7a837cd7" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34017 } }
I20260807 15:15:09.087435 7799 leader_election.cc:302] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f784e211d1f043d0bf78f39f7a837cd7; no voters:
I20260807 15:15:09.087785 7799 leader_election.cc:288] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:09.087921 7805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:09.088176 7805 raft_consensus.cc:686] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [term 1 LEADER]: Becoming Leader. State: Replica: f784e211d1f043d0bf78f39f7a837cd7, State: Running, Role: LEADER
I20260807 15:15:09.088472 7805 consensus_queue.cc:237] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f784e211d1f043d0bf78f39f7a837cd7" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34017 } }
I20260807 15:15:09.089284 7799 sys_catalog.cc:564] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:09.090435 7809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f784e211d1f043d0bf78f39f7a837cd7. Latest consensus state: current_term: 1 leader_uuid: "f784e211d1f043d0bf78f39f7a837cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f784e211d1f043d0bf78f39f7a837cd7" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34017 } } }
I20260807 15:15:09.090447 7808 sys_catalog.cc:455] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f784e211d1f043d0bf78f39f7a837cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f784e211d1f043d0bf78f39f7a837cd7" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34017 } } }
I20260807 15:15:09.090584 7809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:09.090615 7808 sys_catalog.cc:458] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:09.091065 7817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:09.094008 7817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:09.099776 7817 catalog_manager.cc:1347] Generated new cluster ID: b5bf3b362fb3483ebf0b0a5265bc0b9f
I20260807 15:15:09.099850 7817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:09.109866 7817 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:09.110764 7817 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:09.117262 7817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f784e211d1f043d0bf78f39f7a837cd7: Generated new TSK 0
I20260807 15:15:09.118045 7817 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:15:09.145637 7844 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 15:15:09.145732 7844 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:15:09.146533 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:09.130358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.31.230.254:34017)
0807 15:15:09.130514 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:09.130518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:09.130707 (+ 189us) server_negotiation.cc:407] Connection header received
0807 15:15:09.130891 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:09.130893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:09.130943 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:09.131026 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:15:09.131302 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.131857 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.133109 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.133322 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.133398 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.133781 (+ 383us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:15:09.146058 (+ 12277us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:15:09.146369 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53352: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 15:15:09.146597 7803 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53352: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 15:15:09.146723 7840 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:09.130299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34017 (local address 127.0.0.1:53352)
0807 15:15:09.130600 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:09.130615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:09.130744 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:09.131061 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:09.131065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:09.131075 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:15:09.131193 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:09.131200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.131977 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.131980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:09.132991 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:09.133001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.133449 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.133453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:09.133563 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.133592 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:15:09.146311 (+ 12719us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:15:09.146476 (+ 165us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.230.254:34017: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46281/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":118,"threads_started":1}
I20260807 15:15:09.147830 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 7707
2026-08-07T15:15:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (342 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 15:15:09.191394 32667 mini_oidc.cc:152] Starting JWKS server
I20260807 15:15:09.192342 32667 webserver.cc:460] Webserver started at https://127.0.0.1:45145/ using document root <none> and password file <none>
I20260807 15:15:09.192469 32667 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:15:09.192749 32667 webserver.cc:460] Webserver started at http://127.0.0.1:35607/ using document root <none> and password file <none>
2026-08-07T15:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:09Z Disabled control of system clock
I20260807 15:15:09.203320 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33379
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:33379
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45145/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:15:09.322381 7853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:09.322718 7853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:09.322817 7853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:09.326889 7853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:09.327000 7853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:09.327050 7853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:09.327112 7853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:09.327153 7853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:15:09.327201 7853 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:15:09.327278 7853 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45145/jwks.json
I20260807 15:15:09.332230 7853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:33379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45145/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:09.333415 7853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:09.334957 7853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:09.344105 7864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:09.344261 7862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:09.344348 7853 server_base.cc:1034] running on GCE node
W20260807 15:15:09.344545 7861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:09.345203 7853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:09.346401 7853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:09.347605 7853 hybrid_clock.cc:648] HybridClock initialized: now 1786115709347572 us; error 47 us; skew 500 ppm
I20260807 15:15:09.349599 7853 webserver.cc:460] Webserver started at http://127.31.230.254:33871/ using document root <none> and password file <none>
I20260807 15:15:09.350203 7853 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:09.350301 7853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:09.350541 7853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:09.352723 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "f5d13b8afe2d4903bc69dc72acd33b15"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "633d0b76c4826d39be76e3732e0beec8"
server_key_iv: "5d6ab1136f380c7c5593680953034c0d"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.353312 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "f5d13b8afe2d4903bc69dc72acd33b15"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "633d0b76c4826d39be76e3732e0beec8"
server_key_iv: "5d6ab1136f380c7c5593680953034c0d"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.357131 7853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:09.359748 7875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:09.361096 7853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:15:09.361258 7853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "f5d13b8afe2d4903bc69dc72acd33b15"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "633d0b76c4826d39be76e3732e0beec8"
server_key_iv: "5d6ab1136f380c7c5593680953034c0d"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.361379 7853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:09.383750 7853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:09.384528 7853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:09.384758 7853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:09.392243 7853 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:33379
I20260807 15:15:09.392259 7941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:33379 every 8 connection(s)
I20260807 15:15:09.393345 7853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:09.396628 7942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:09.400138 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 7853
I20260807 15:15:09.400297 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:09.400584 32667 external_mini_cluster.cc:1428] Setting key 4917215ceea84713945cc9590421c4e2
I20260807 15:15:09.402683 7942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15: Bootstrap starting.
I20260807 15:15:09.404904 7942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:09.405743 7942 log.cc:824] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:09.407747 7942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15: No bootstrap required, opened a new log
I20260807 15:15:09.410305 7949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:09.402062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.31.230.254:33379)
0807 15:15:09.402720 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:09.402735 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:09.402771 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:15:09.403442 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:09.403460 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:09.403847 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:09.404195 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:09.404650 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.405782 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.406917 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.407399 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.407579 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.408507 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:09.408512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:09.408525 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:09.408908 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:09.408963 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:09.408984 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:09.409096 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:09.409251 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 15:15:09.409503 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260807 15:15:09.411811 7942 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33379 } }
I20260807 15:15:09.412055 7942 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:09.412148 7942 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d13b8afe2d4903bc69dc72acd33b15, State: Initialized, Role: FOLLOWER
I20260807 15:15:09.412583 7942 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33379 } }
I20260807 15:15:09.412744 7942 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:09.412842 7942 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:09.412946 7942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:09.413632 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:09.414100 7942 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33379 } }
I20260807 15:15:09.414471 7942 leader_election.cc:302] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5d13b8afe2d4903bc69dc72acd33b15; no voters:
I20260807 15:15:09.414860 7942 leader_election.cc:288] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:09.414942 7952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:09.415172 7952 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [term 1 LEADER]: Becoming Leader. State: Replica: f5d13b8afe2d4903bc69dc72acd33b15, State: Running, Role: LEADER
I20260807 15:15:09.415472 7952 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33379 } }
I20260807 15:15:09.416200 7942 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:09.416673 7952 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5d13b8afe2d4903bc69dc72acd33b15. Latest consensus state: current_term: 1 leader_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33379 } } }
I20260807 15:15:09.416791 7952 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:09.417204 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d13b8afe2d4903bc69dc72acd33b15" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33379 } } }
I20260807 15:15:09.417371 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:09.417774 7957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:09.420994 7957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:09.426951 7957 catalog_manager.cc:1347] Generated new cluster ID: c750c6c8e3a24852908edf2b2354ebf3
I20260807 15:15:09.427047 7957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:09.440632 7957 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:09.441990 7957 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:09.458580 7957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5d13b8afe2d4903bc69dc72acd33b15: Generated new TSK 0
I20260807 15:15:09.459656 7957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:09.481228 7949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:09.476086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.31.230.254:33379)
0807 15:15:09.476251 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:09.476256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:09.476442 (+ 186us) server_negotiation.cc:407] Connection header received
0807 15:15:09.476609 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:09.476612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:09.476663 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:09.476770 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:15:09.477033 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.477687 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.478923 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.479142 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.479214 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.479555 (+ 341us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:15:09.480820 (+ 1265us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45145/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:15:09.481079 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45145/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 15:15:09.481321 7986 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:09.476009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33379 (local address 127.0.0.1:49638)
0807 15:15:09.476338 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:09.476354 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:09.476476 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:09.476785 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:09.476788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:09.476796 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:15:09.476919 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:09.476926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.477805 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.477809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:09.478798 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:09.478809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.479259 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.479262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:09.479355 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.479385 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:15:09.480988 (+ 1603us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45145/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:15:09.481132 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.230.254:33379: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45145/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":123,"threads_started":1}
W20260807 15:15:09.481320 7949 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45145/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260807 15:15:09.482013 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 7853
E20260807 15:15:09.503273 7987 webserver.cc:533] Webserver: sslize failed:
E20260807 15:15:09.503403 7987 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T15:15:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (341 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T15:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:09Z Disabled control of system clock
I20260807 15:15:09.543294 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33615
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:37355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:33615
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/testchainca.cert with env {}
W20260807 15:15:09.661715 7991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:09.662037 7991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:09.662127 7991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:09.666148 7991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:09.666229 7991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:09.666257 7991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:09.666311 7991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:09.666365 7991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:15:09.666405 7991 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:15:09.666437 7991 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 15:15:09.674340 7991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:33615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:09.675823 7991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:09.677575 7991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:09.687426 8001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:09.687417 7999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:09.687611 7991 server_base.cc:1034] running on GCE node
W20260807 15:15:09.687381 8004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:09.688174 7991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:09.689311 7991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:09.690501 7991 hybrid_clock.cc:648] HybridClock initialized: now 1786115709690495 us; error 71 us; skew 500 ppm
I20260807 15:15:09.692463 7991 webserver.cc:460] Webserver started at http://127.31.230.254:33699/ using document root <none> and password file <none>
I20260807 15:15:09.693086 7991 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:09.693185 7991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:09.693426 7991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:09.695566 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "e8a5c816823041ff8b023ed3ba0cc774"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "138c403ff147605dcef34bfd7a3e0751"
server_key_iv: "1faaa3e9c3621eba90205c79d34d0443"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.696170 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "e8a5c816823041ff8b023ed3ba0cc774"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "138c403ff147605dcef34bfd7a3e0751"
server_key_iv: "1faaa3e9c3621eba90205c79d34d0443"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.700201 7991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:09.702948 8013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:09.704345 7991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:09.704532 7991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "e8a5c816823041ff8b023ed3ba0cc774"
format_stamp: "Formatted at 2026-08-07 15:15:09 on dist-test-slave-gkw7"
server_key: "138c403ff147605dcef34bfd7a3e0751"
server_key_iv: "1faaa3e9c3621eba90205c79d34d0443"
server_key_version: "encryptionkey@0"
I20260807 15:15:09.704675 7991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:09.734959 7991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:09.735858 7991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:09.736130 7991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:09.744942 7991 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:33615
I20260807 15:15:09.744957 8086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:33615 every 8 connection(s)
I20260807 15:15:09.746124 7991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:09.749166 8087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:09.750716 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 7991
I20260807 15:15:09.750854 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:09.751161 32667 external_mini_cluster.cc:1428] Setting key 39a66a15db6d4a77e4d961d750142d7b
I20260807 15:15:09.755712 8087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774: Bootstrap starting.
I20260807 15:15:09.758147 8087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:09.758919 8087 log.cc:824] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:09.760968 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:09.752691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.31.230.254:33615)
0807 15:15:09.753374 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:09.753390 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:09.753428 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:15:09.754173 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:09.754193 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:09.754611 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:09.754965 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:09.755411 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.756444 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.757385 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.757938 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.758122 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.759075 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:09.759082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:09.759100 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:09.759634 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:09.759711 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:09.759742 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:09.759843 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:09.759963 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 15:15:09.760239 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":142,"threads_started":1}
I20260807 15:15:09.761152 8087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774: No bootstrap required, opened a new log
I20260807 15:15:09.764266 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:09.765450 8087 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a5c816823041ff8b023ed3ba0cc774" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33615 } }
I20260807 15:15:09.765775 8087 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:09.765858 8087 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a5c816823041ff8b023ed3ba0cc774, State: Initialized, Role: FOLLOWER
I20260807 15:15:09.766323 8087 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a5c816823041ff8b023ed3ba0cc774" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33615 } }
I20260807 15:15:09.766481 8087 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:09.766572 8087 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:09.766677 8087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:09.767673 8087 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a5c816823041ff8b023ed3ba0cc774" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33615 } }
I20260807 15:15:09.768033 8087 leader_election.cc:302] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8a5c816823041ff8b023ed3ba0cc774; no voters:
I20260807 15:15:09.768376 8087 leader_election.cc:288] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:09.768504 8095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:09.768791 8095 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [term 1 LEADER]: Becoming Leader. State: Replica: e8a5c816823041ff8b023ed3ba0cc774, State: Running, Role: LEADER
I20260807 15:15:09.769109 8095 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a5c816823041ff8b023ed3ba0cc774" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33615 } }
I20260807 15:15:09.769804 8087 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:09.771123 8096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8a5c816823041ff8b023ed3ba0cc774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a5c816823041ff8b023ed3ba0cc774" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33615 } } }
I20260807 15:15:09.771124 8097 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8a5c816823041ff8b023ed3ba0cc774. Latest consensus state: current_term: 1 leader_uuid: "e8a5c816823041ff8b023ed3ba0cc774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a5c816823041ff8b023ed3ba0cc774" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33615 } } }
I20260807 15:15:09.771296 8096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:09.771298 8097 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:09.771677 8105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:09.775210 8105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:09.781263 8105 catalog_manager.cc:1347] Generated new cluster ID: 0b0dd24bca0d412a86f32f963e89edb0
I20260807 15:15:09.781359 8105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:09.788341 8105 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:09.789592 8105 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:09.794234 8105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8a5c816823041ff8b023ed3ba0cc774: Generated new TSK 0
I20260807 15:15:09.794992 8105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:09.942582 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:09.807639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.31.230.254:33615)
0807 15:15:09.807770 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:09.807775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:09.807970 (+ 195us) server_negotiation.cc:407] Connection header received
0807 15:15:09.808232 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:09.808243 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:09.808295 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:09.808419 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:15:09.808674 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.809392 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.810323 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.810558 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.810681 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.811021 (+ 340us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:15:09.941833 (+130812us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:15:09.942387 (+ 554us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47248: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 15:15:09.942708 8091 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47248: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 15:15:09.942761 8126 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:09.807559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33615 (local address 127.0.0.1:47248)
0807 15:15:09.807876 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:09.807889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:09.808022 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:09.808418 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:09.808421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:09.808429 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:15:09.808546 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:09.808553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.809547 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.809552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:09.810194 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:09.810202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:09.810699 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:09.810703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:09.810808 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:09.810832 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:15:09.942250 (+131418us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:15:09.942494 (+ 244us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.230.254:33615: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":113,"threads_started":1}
I20260807 15:15:09.943564 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 7991
2026-08-07T15:15:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (435 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (127 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:10 dist-test-slave-gkw7 krb5kdc[8146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:10 dist-test-slave-gkw7 krb5kdc[8146](info): set up 2 sockets
Aug 07 15:15:10 dist-test-slave-gkw7 krb5kdc[8146](info): commencing operation
krb5kdc: starting...
W20260807 15:15:11.482085 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:11 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:11.616766 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37019
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:42537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:11.734030 8161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:11.734378 8161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:11.734462 8161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:11.738539 8161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:11.738647 8161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:11.738684 8161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:11.738739 8161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:11.738776 8161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:11.744257 8161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:42537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:11.745476 8161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:11.746624 8161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:11.755621 8170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.755662 8173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.755656 8167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.755877 8161 server_base.cc:1034] running on GCE node
I20260807 15:15:11.756526 8161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:11.757838 8161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:11.759033 8161 hybrid_clock.cc:648] HybridClock initialized: now 1786115711759012 us; error 40 us; skew 500 ppm
Aug 07 15:15:11 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:11.762068 8161 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:11.763000 8161 webserver.cc:460] Webserver started at http://127.31.230.254:45805/ using document root <none> and password file <none>
I20260807 15:15:11.763597 8161 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:11.763695 8161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:11.763960 8161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:11.766181 8161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "f7094af0019d4a7c942f3bb2a4f271e9"
format_stamp: "Formatted at 2026-08-07 15:15:11 on dist-test-slave-gkw7"
server_key: "61a1b11c35d0cbe4d6280356547d9b12"
server_key_iv: "912fc9624c1a1669691215bd74f18cab"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.766813 8161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "f7094af0019d4a7c942f3bb2a4f271e9"
format_stamp: "Formatted at 2026-08-07 15:15:11 on dist-test-slave-gkw7"
server_key: "61a1b11c35d0cbe4d6280356547d9b12"
server_key_iv: "912fc9624c1a1669691215bd74f18cab"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.770797 8161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:11.773350 8182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.774712 8161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:11.774873 8161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "f7094af0019d4a7c942f3bb2a4f271e9"
format_stamp: "Formatted at 2026-08-07 15:15:11 on dist-test-slave-gkw7"
server_key: "61a1b11c35d0cbe4d6280356547d9b12"
server_key_iv: "912fc9624c1a1669691215bd74f18cab"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.774995 8161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:11.794049 8161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:11.797752 8161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:11.797984 8161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:11.805758 8161 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:37019
I20260807 15:15:11.805806 8256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:37019 every 8 connection(s)
I20260807 15:15:11.806831 8161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:11.810060 8257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:11.814877 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 8161
I20260807 15:15:11.814994 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:11.815279 32667 external_mini_cluster.cc:1428] Setting key 4b8b9b361ffae1cefc02297c7e57b138
I20260807 15:15:11.816641 8257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9: Bootstrap starting.
I20260807 15:15:11.819696 8257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:11.820803 8257 log.cc:824] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:11.823143 8257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9: No bootstrap required, opened a new log
Aug 07 15:15:11 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:11.827769 8257 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37019 } }
I20260807 15:15:11.828019 8257 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:11.828107 8257 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7094af0019d4a7c942f3bb2a4f271e9, State: Initialized, Role: FOLLOWER
I20260807 15:15:11.828722 8257 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37019 } }
I20260807 15:15:11.828933 8257 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:11.829092 8257 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:11.829231 8257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:11.830428 8257 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37019 } }
I20260807 15:15:11.830854 8257 leader_election.cc:302] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7094af0019d4a7c942f3bb2a4f271e9; no voters:
I20260807 15:15:11.831264 8257 leader_election.cc:288] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:11.831360 8262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:11.831553 8262 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [term 1 LEADER]: Becoming Leader. State: Replica: f7094af0019d4a7c942f3bb2a4f271e9, State: Running, Role: LEADER
I20260807 15:15:11.831890 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.816840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.31.230.254:37019)
0807 15:15:11.817315 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:11.817338 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:11.817385 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:15:11.818366 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:11.818392 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:11.818840 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:11.819234 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:11.819997 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:11.821525 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:11.822493 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:11.823237 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:11.823419 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:11.826455 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:11.826460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:11.826474 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:11.826505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:11.828460 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:11.829026 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:11.829033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:11.829040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:11.829175 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:11.829518 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:11.829521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:11.829522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:11.829995 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:11.830154 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:11.830514 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 15:15:11.830877 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260807 15:15:11.831933 8262 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37019 } }
I20260807 15:15:11.832690 8257 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:11.834004 8265 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7094af0019d4a7c942f3bb2a4f271e9. Latest consensus state: current_term: 1 leader_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37019 } } }
I20260807 15:15:11.834216 8265 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:11.834539 8264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7094af0019d4a7c942f3bb2a4f271e9" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37019 } } }
I20260807 15:15:11.834697 8264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:11.834767 8275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:11.838384 8275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:11.844445 8275 catalog_manager.cc:1347] Generated new cluster ID: aa049f91c187451191d315c7773f9f07
I20260807 15:15:11.844523 8275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:11.867427 8275 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:11.868808 8275 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:11.878501 8275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7094af0019d4a7c942f3bb2a4f271e9: Generated new TSK 0
I20260807 15:15:11.879436 8275 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:15:11.921072 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37019
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:42537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:12.041282 8289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:12.041651 8289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:12.041703 8289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:12.045802 8289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:12.045869 8289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:12.045930 8289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:12.051524 8289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:42537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:12.052547 8289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:12.053625 8289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:12.063892 8289 server_base.cc:1034] running on GCE node
W20260807 15:15:12.063840 8295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:12.063853 8297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:12.063855 8300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:12.064539 8289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:12.065145 8289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:12.066370 8289 hybrid_clock.cc:648] HybridClock initialized: now 1786115712066354 us; error 29 us; skew 500 ppm
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.069339 8289 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:15:12.070343 8289 webserver.cc:460] Webserver started at http://127.31.230.193:38079/ using document root <none> and password file <none>
I20260807 15:15:12.070894 8289 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:12.070953 8289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:12.071139 8289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:12.073279 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "2ef6e4ac7129449c9a0d45c04169f387"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "78c2e1fd0682c369bf1b598748b98c23"
server_key_iv: "6d4f5caffc843b542c3e3f623c071f3c"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.073890 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "2ef6e4ac7129449c9a0d45c04169f387"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "78c2e1fd0682c369bf1b598748b98c23"
server_key_iv: "6d4f5caffc843b542c3e3f623c071f3c"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.077860 8289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:12.080618 8310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.081947 8289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:12.082047 8289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "2ef6e4ac7129449c9a0d45c04169f387"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "78c2e1fd0682c369bf1b598748b98c23"
server_key_iv: "6d4f5caffc843b542c3e3f623c071f3c"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.082135 8289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:12.100915 8289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:12.104079 8289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:12.104306 8289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:12.104929 8289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:12.105856 8289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:12.105906 8289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.105976 8289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:12.106006 8289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.115239 8289 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:35215
I20260807 15:15:12.115265 8464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:35215 every 8 connection(s)
I20260807 15:15:12.116214 8289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:12.118958 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 8289
I20260807 15:15:12.119093 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:12.119398 32667 external_mini_cluster.cc:1428] Setting key 52e8cbd72ca8e943953173ad6293a609
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:15:12.132673 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.118173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:58997 (local address 127.31.230.254:37019)
0807 15:15:12.118356 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.118361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.119334 (+ 973us) server_negotiation.cc:407] Connection header received
0807 15:15:12.121003 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.121006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.121059 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.121148 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.122462 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.123345 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.125153 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.125624 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.125768 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.128470 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:12.128477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:12.128481 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:12.128521 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:12.130409 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.130983 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.130989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.130994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.131096 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.131453 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.131456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.131461 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.131748 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:12.131864 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:12.132376 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 15:15:12.132525 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:15:12.133781 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.118603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37019 (local address 127.31.230.193:58997)
0807 15:15:12.119163 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:12.119210 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:12.120749 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:12.121320 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:12.121327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:12.121793 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:12.122296 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:12.122310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.123512 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.123517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:12.125022 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:12.125033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.125784 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.125787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:12.125945 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.126609 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:12.126634 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:12.128277 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:12.130548 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:12.130554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:12.130569 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:12.130865 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:12.131203 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:12.131205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:12.131206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:12.131347 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:12.131867 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:12.131872 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:12.132138 (+ 266us) client_negotiation.cc:769] Sending connection context
0807 15:15:12.132319 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 15:15:12.132752 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":153,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:15:12.135228 8466 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37019
I20260807 15:15:12.135509 8466 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:12.136039 8466 heartbeater.cc:511] Master 127.31.230.254:37019 requested a full tablet report, sending...
I20260807 15:15:12.137959 8204 ts_manager.cc:194] Registered new tserver with Master: 2ef6e4ac7129449c9a0d45c04169f387 (127.31.230.193:35215)
I20260807 15:15:12.139140 8204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:58997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:15:12.155171 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37019
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:42537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:12.272042 8476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:12.272552 8476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:12.272680 8476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:12.276981 8476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:12.277051 8476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:12.277148 8476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:12.282938 8476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:42537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:12.284210 8476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:12.285348 8476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:12.295630 8486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:12.295650 8482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:12.295658 8484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:12.296058 8476 server_base.cc:1034] running on GCE node
I20260807 15:15:12.296485 8476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:12.297204 8476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:12.298401 8476 hybrid_clock.cc:648] HybridClock initialized: now 1786115712298351 us; error 63 us; skew 500 ppm
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.301398 8476 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:15:12.302374 8476 webserver.cc:460] Webserver started at http://127.31.230.194:35143/ using document root <none> and password file <none>
I20260807 15:15:12.302989 8476 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:12.303088 8476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:12.303339 8476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:12.305437 8476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "f3dc3d8562414af0ad0728d7d8f9ca6a"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "81455ba063af8eac8ca25cb47e8b776c"
server_key_iv: "d1c3c06c035e3248249d43f5f2efadba"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.306092 8476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "f3dc3d8562414af0ad0728d7d8f9ca6a"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "81455ba063af8eac8ca25cb47e8b776c"
server_key_iv: "d1c3c06c035e3248249d43f5f2efadba"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.310495 8476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:12.313079 8496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.314851 8476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:12.314988 8476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "f3dc3d8562414af0ad0728d7d8f9ca6a"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "81455ba063af8eac8ca25cb47e8b776c"
server_key_iv: "d1c3c06c035e3248249d43f5f2efadba"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.315132 8476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:12.357795 8476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:12.361177 8476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:12.361414 8476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:12.362125 8476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:12.363077 8476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:12.363153 8476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.363229 8476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:12.363281 8476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.372474 8476 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:34757
I20260807 15:15:12.372495 8659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:34757 every 8 connection(s)
I20260807 15:15:12.373440 8476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:12.383445 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 8476
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:12.383546 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:12.383828 32667 external_mini_cluster.cc:1428] Setting key ab6f718a4985a486a688769e54a15d46
I20260807 15:15:12.388559 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.375375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:41055 (local address 127.31.230.254:37019)
0807 15:15:12.375513 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.375516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.376339 (+ 823us) server_negotiation.cc:407] Connection header received
0807 15:15:12.377857 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.377861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.377920 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.377999 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.379400 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.380039 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.381213 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.381465 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.381624 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.384200 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:12.384207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:12.384210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:12.384243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:12.386054 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.386752 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.386755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.386756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.386837 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.387241 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.387244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.387246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.387428 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:12.387505 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:12.388189 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 15:15:12.388342 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:15:12.389356 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.375699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37019 (local address 127.31.230.194:41055)
0807 15:15:12.376178 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:12.376223 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:12.377622 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:12.378202 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:12.378211 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:12.378713 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:12.379247 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:12.379263 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.380171 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.380177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:12.381078 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:12.381088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.381670 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.381674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:12.381863 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.382435 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:12.382451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:12.384022 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:12.386202 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:12.386211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:12.386231 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:12.386641 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:12.386932 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:12.386938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:12.386943 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:12.387142 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:12.387534 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:12.387543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:12.387908 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 15:15:12.388102 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 15:15:12.388383 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":107,"threads_started":1}
I20260807 15:15:12.390823 8660 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37019
I20260807 15:15:12.391120 8660 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:12.391664 8660 heartbeater.cc:511] Master 127.31.230.254:37019 requested a full tablet report, sending...
I20260807 15:15:12.392881 8204 ts_manager.cc:194] Registered new tserver with Master: f3dc3d8562414af0ad0728d7d8f9ca6a (127.31.230.194:34757)
I20260807 15:15:12.393800 8204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:41055
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:15:12.417007 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37019
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:42537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:12.534602 8666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:12.534911 8666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:12.534991 8666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:12.539121 8666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:12.539223 8666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:12.539323 8666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:12.544622 8666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:42537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:12.545820 8666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:12.546919 8666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:12.556779 8673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:12.556777 8675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:12.556777 8672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:12.557262 8666 server_base.cc:1034] running on GCE node
I20260807 15:15:12.557713 8666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:12.558360 8666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:12.559556 8666 hybrid_clock.cc:648] HybridClock initialized: now 1786115712559540 us; error 28 us; skew 500 ppm
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.565379 8666 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:15:12.566335 8666 webserver.cc:460] Webserver started at http://127.31.230.195:34637/ using document root <none> and password file <none>
I20260807 15:15:12.566875 8666 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:12.566934 8666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:12.567109 8666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:12.569242 8666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "dd480722415a45a89cf374fbea26224a"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "41b155bafcd5c10df5e4c9949276d2d4"
server_key_iv: "067caad6ac1e55f5370a1f6f256584b0"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.569852 8666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "dd480722415a45a89cf374fbea26224a"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "41b155bafcd5c10df5e4c9949276d2d4"
server_key_iv: "067caad6ac1e55f5370a1f6f256584b0"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.573700 8666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:15:12.576282 8687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.577648 8666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:12.577771 8666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "dd480722415a45a89cf374fbea26224a"
format_stamp: "Formatted at 2026-08-07 15:15:12 on dist-test-slave-gkw7"
server_key: "41b155bafcd5c10df5e4c9949276d2d4"
server_key_iv: "067caad6ac1e55f5370a1f6f256584b0"
server_key_version: "encryptionkey@0"
I20260807 15:15:12.577886 8666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:12.597055 8666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:12.600628 8666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:12.600853 8666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:12.601508 8666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:12.602461 8666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:12.602519 8666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.602588 8666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:12.602627 8666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:12.613061 8666 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:34235
I20260807 15:15:12.613132 8844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:34235 every 8 connection(s)
I20260807 15:15:12.614156 8666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:12.614863 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 8666
I20260807 15:15:12.615013 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:12.615356 32667 external_mini_cluster.cc:1428] Setting key 6b9b7f90d6ffeb27dfcee3beb85cf8fe
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:12.630702 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.616527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:48165 (local address 127.31.230.254:37019)
0807 15:15:12.616691 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.616695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.617969 (+ 1274us) server_negotiation.cc:407] Connection header received
0807 15:15:12.619783 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.619787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.619864 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.619974 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.621390 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.622118 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.623282 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.623528 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.623625 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.626491 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:12.626494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:12.626496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:12.626525 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:12.628006 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.628679 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.628683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.628685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.628793 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.629272 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.629276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.629277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.629610 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:12.629720 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:12.630431 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 15:15:12.630556 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:15:12.631603 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.617120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37019 (local address 127.31.230.195:48165)
0807 15:15:12.617799 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:12.617864 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:12.619421 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:12.620211 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:12.620222 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:12.620698 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:12.621217 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:12.621231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.622248 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.622252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:12.623152 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:12.623161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.623672 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.623675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:12.623836 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.624542 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:12.624559 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:12.626259 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:12.628124 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:12.628130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:12.628146 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:12.628552 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:12.628923 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:12.628926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:12.628927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:12.629144 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:12.629729 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:12.629734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:12.630132 (+ 398us) client_negotiation.cc:769] Sending connection context
0807 15:15:12.630376 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 15:15:12.630679 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":168,"threads_started":1}
I20260807 15:15:12.632928 8845 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37019
I20260807 15:15:12.633193 8845 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:12.633834 8845 heartbeater.cc:511] Master 127.31.230.254:37019 requested a full tablet report, sending...
I20260807 15:15:12.635010 8204 ts_manager.cc:194] Registered new tserver with Master: dd480722415a45a89cf374fbea26224a (127.31.230.195:34235)
I20260807 15:15:12.635659 8204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:48165
I20260807 15:15:12.642359 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/test-admin.keytab.
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.662256 32667 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 15:15:12.665220 8854 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:12.663820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.31.230.193:35215)
0807 15:15:12.664121 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.664127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.664153 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:15:12.664209 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.664214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.664416 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.664546 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.665048 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34932: BlockingRecv error: recv got EOF from 127.0.0.1:34932 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":91,"threads_started":1}
W20260807 15:15:12.665710 32667 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.666823 32667 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.667773 32667 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:15:12.678334 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.668811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.31.230.193:35215)
0807 15:15:12.668923 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.668927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.668964 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:15:12.669078 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.669080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.669123 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.669191 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.669795 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.670547 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.671553 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.671956 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.672041 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.674833 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:12.674844 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:12.674861 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:12.674886 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:12.676446 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.676903 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.676909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.676910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.676993 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.677232 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.677234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.677235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.677487 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:12.677811 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:12.677985 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:15:12.678193 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 15:15:12.682559 8854 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:12.681395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.31.230.193:35215)
0807 15:15:12.681521 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.681543 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.681700 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:15:12.681883 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.681887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.681935 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.682017 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.682445 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34956: BlockingRecv error: recv got EOF from 127.0.0.1:34956 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:15:12.682687 32667 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.683787 32667 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.684803 32667 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:15:12.693967 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.685796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.31.230.193:35215)
0807 15:15:12.685908 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.685914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.686116 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:15:12.686266 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.686270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.686318 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.686404 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.686932 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.687443 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.688252 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.688497 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.688608 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.690909 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:12.690912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:12.690914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:12.690938 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:12.692424 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.692807 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.692810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.692811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.692887 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.693114 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.693116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.693117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.693298 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:12.693415 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:12.693652 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 15:15:12.693753 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 15:15:12.696588 8854 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:12.695735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.31.230.193:35215)
0807 15:15:12.695880 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.695883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.695895 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:15:12.696009 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.696011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.696045 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.696108 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.696472 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34978: BlockingRecv error: recv got EOF from 127.0.0.1:34978 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 15:15:12.696666 32667 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115680745020-32667-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.697793 32667 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:12.698727 32667 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:15:12.707696 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.699550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.31.230.193:35215)
0807 15:15:12.699681 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:12.699687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:12.699777 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:15:12.699921 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:12.699924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:12.699966 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:12.700058 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:12.700572 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.701083 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.701976 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:12.702227 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:12.702342 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:12.704630 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:12.704634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:12.704637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:12.704670 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:12.706005 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.706444 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.706448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.706450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.706533 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:12.706821 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:12.706825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:12.706827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:12.707009 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:12.707170 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:12.707408 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 15:15:12.707520 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:15:12.708827 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 8289
I20260807 15:15:12.734522 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 8476
I20260807 15:15:12.745723 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 8666
I20260807 15:15:12.758702 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 8161
2026-08-07T15:15:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2558 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8873](info): set up 2 sockets
Aug 07 15:15:12 dist-test-slave-gkw7 krb5kdc[8873](info): commencing operation
krb5kdc: starting...
W20260807 15:15:14.092661 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab.
Entry for principal oryx/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab.
I20260807 15:15:14.228374 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37793
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab
--principal=oryx/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:14.347208 8895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:14.347522 8895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:14.347599 8895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:14.352288 8895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:14.352389 8895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:14.352424 8895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:14.352463 8895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:14.352499 8895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:14.357766 8895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab
--principal=oryx/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:14.358903 8895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:14.359963 8895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:14.369050 8906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:14.369108 8901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:14.369055 8900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:14.369320 8895 server_base.cc:1034] running on GCE node
I20260807 15:15:14.369984 8895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:14.371104 8895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:14.372303 8895 hybrid_clock.cc:648] HybridClock initialized: now 1786115714372283 us; error 37 us; skew 500 ppm
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:14.375290 8895 init.cc:377] Logged in from keytab as oryx/127.31.230.254@KRBTEST.COM (short username oryx)
I20260807 15:15:14.376338 8895 webserver.cc:460] Webserver started at http://127.31.230.254:46069/ using document root <none> and password file <none>
I20260807 15:15:14.376958 8895 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:14.377020 8895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:14.377290 8895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:14.379457 8895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "1ce63408ebb0481eb74bd908e2211fba"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "489f3c927c1594eafe8d54a2923a1292"
server_key_iv: "d10b6bc6703598ce15ffaa33b12fb998"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.380057 8895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "1ce63408ebb0481eb74bd908e2211fba"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "489f3c927c1594eafe8d54a2923a1292"
server_key_iv: "d10b6bc6703598ce15ffaa33b12fb998"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.383905 8895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:14.386387 8914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.387643 8895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:14.387806 8895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "1ce63408ebb0481eb74bd908e2211fba"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "489f3c927c1594eafe8d54a2923a1292"
server_key_iv: "d10b6bc6703598ce15ffaa33b12fb998"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.387899 8895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:14.408362 8895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:14.411864 8895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:14.412091 8895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:14.419507 8895 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:37793
I20260807 15:15:14.419817 8983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:37793 every 8 connection(s)
I20260807 15:15:14.420535 8895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:14.423388 8984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:14.425773 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 8895
I20260807 15:15:14.425930 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:14.426237 32667 external_mini_cluster.cc:1428] Setting key 62b516b8563fbec0d4a77e88b81038b8
I20260807 15:15:14.429677 8984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Bootstrap starting.
I20260807 15:15:14.432354 8984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:14.433205 8984 log.cc:824] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:14.435786 8984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: No bootstrap required, opened a new log
I20260807 15:15:14.440860 8984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } }
I20260807 15:15:14.441152 8984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:14.441215 8984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ce63408ebb0481eb74bd908e2211fba, State: Initialized, Role: FOLLOWER
I20260807 15:15:14.441782 8984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } }
I20260807 15:15:14.441953 8984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:14.441995 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.427804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.31.230.254:37793)
0807 15:15:14.428241 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:14.428257 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:14.428294 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:15:14.428940 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:14.428957 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:14.429293 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:14.429647 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:14.430543 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.431567 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.432777 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.433260 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.433465 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:14.436430 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:14.436437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:14.436456 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:14.436489 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:14.438639 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:14.439096 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:14.439104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:14.439114 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:14.439277 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:14.439554 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:14.439557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:14.439559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:14.440127 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:14.440355 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:14.440557 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:15:14.440875 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":146,"threads_started":1}
I20260807 15:15:14.442032 8984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:14.442123 8984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:14.443368 8984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } }
I20260807 15:15:14.443732 8984 leader_election.cc:302] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ce63408ebb0481eb74bd908e2211fba; no voters:
I20260807 15:15:14.444072 8984 leader_election.cc:288] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:14.444188 8993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:14.444410 8993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 LEADER]: Becoming Leader. State: Replica: 1ce63408ebb0481eb74bd908e2211fba, State: Running, Role: LEADER
I20260807 15:15:14.444721 8993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } }
I20260807 15:15:14.445359 8984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:14.446557 8994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } }
I20260807 15:15:14.446728 8994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:14.446555 8995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ce63408ebb0481eb74bd908e2211fba. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } }
I20260807 15:15:14.446941 8995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:14.447182 9002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:14.450515 9002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:14.456524 9002 catalog_manager.cc:1347] Generated new cluster ID: 2ff8203034124349b90f11131fc8c229
I20260807 15:15:14.456595 9002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:14.484954 9002 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:14.486435 9002 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:14.493502 9002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Generated new TSK 0
I20260807 15:15:14.494253 9002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab.
Entry for principal oryx/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab.
I20260807 15:15:14.541852 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37793
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:14.676813 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:14.677088 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:14.677132 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:14.681173 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:14.681236 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:14.681339 9018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:14.691125 9018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:14.692524 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:14.693922 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:14.703958 9032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:14.704035 9026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:14.703966 9027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:14.704211 9018 server_base.cc:1034] running on GCE node
I20260807 15:15:14.704682 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:14.705370 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:14.706599 9018 hybrid_clock.cc:648] HybridClock initialized: now 1786115714706575 us; error 42 us; skew 500 ppm
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:14.709491 9018 init.cc:377] Logged in from keytab as oryx/127.31.230.193@KRBTEST.COM (short username oryx)
I20260807 15:15:14.710464 9018 webserver.cc:460] Webserver started at http://127.31.230.193:34851/ using document root <none> and password file <none>
I20260807 15:15:14.711062 9018 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:14.711160 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:14.711410 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:14.713471 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "ded878401a1f46989497aec764dc1942"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "c204e2f5ecc9d56dc91aac1b115db771"
server_key_iv: "09e12ab25fc7729277d360fdaa93f428"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.714129 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "ded878401a1f46989497aec764dc1942"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "c204e2f5ecc9d56dc91aac1b115db771"
server_key_iv: "09e12ab25fc7729277d360fdaa93f428"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.717918 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:14.720412 9043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.721666 9018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:14.721810 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "ded878401a1f46989497aec764dc1942"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "c204e2f5ecc9d56dc91aac1b115db771"
server_key_iv: "09e12ab25fc7729277d360fdaa93f428"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.721928 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:14.734625 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:14.737820 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:14.738032 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:14.738638 9018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:14.739529 9018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:14.739603 9018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.739677 9018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:14.739729 9018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.749225 9018 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:39951
I20260807 15:15:14.749253 9198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:39951 every 8 connection(s)
I20260807 15:15:14.750182 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:14.753954 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 9018
I20260807 15:15:14.754104 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:14.754421 32667 external_mini_cluster.cc:1428] Setting key e82ec8dfc6e3ff47e33086313b779d5b
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:15:14.766463 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.752197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:33257 (local address 127.31.230.254:37793)
0807 15:15:14.752375 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:14.752380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:14.753219 (+ 839us) server_negotiation.cc:407] Connection header received
0807 15:15:14.754819 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:14.754822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:14.754883 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:14.755001 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:14.756471 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.757344 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.758992 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.759225 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.759320 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:14.762306 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:14.762313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:14.762319 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:14.762358 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:14.764063 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:14.764603 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:14.764608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:14.764610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:14.764712 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:14.765061 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:14.765065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:14.765067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:14.765375 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:14.765497 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:14.766055 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 15:15:14.766214 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:15:14.767246 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.752602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:33257)
0807 15:15:14.753046 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:14.753102 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:14.754605 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:14.755191 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:14.755198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:14.755567 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:14.756296 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:14.756311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.757489 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.757491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:14.758845 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:14.758861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.759408 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.759415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:14.759647 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:14.760337 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:14.760354 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:14.762109 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:14.764181 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:14.764186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:14.764200 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:14.764473 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:14.764813 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:14.764815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:14.764816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:14.764950 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:14.765499 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:14.765506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:14.765783 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 15:15:14.765940 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 15:15:14.766273 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
I20260807 15:15:14.768709 9199 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.194@KRBTEST.COM" created.
I20260807 15:15:14.769025 9199 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:14.769686 9199 heartbeater.cc:511] Master 127.31.230.254:37793 requested a full tablet report, sending...
I20260807 15:15:14.771536 8934 ts_manager.cc:194] Registered new tserver with Master: ded878401a1f46989497aec764dc1942 (127.31.230.193:39951)
I20260807 15:15:14.772907 8934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:33257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab.
Entry for principal oryx/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab.
I20260807 15:15:14.784996 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37793
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:14.898452 9204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:14.898763 9204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:14.898846 9204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:14.902992 9204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:14.903082 9204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:14.903172 9204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:14.908560 9204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:14.909796 9204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:14.910867 9204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:14.920822 9216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:14.920881 9204 server_base.cc:1034] running on GCE node
W20260807 15:15:14.920817 9212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:14.920822 9213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:14.922109 9204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:14.922955 9204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:14.924132 9204 hybrid_clock.cc:648] HybridClock initialized: now 1786115714924101 us; error 59 us; skew 500 ppm
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:14.927156 9204 init.cc:377] Logged in from keytab as oryx/127.31.230.194@KRBTEST.COM (short username oryx)
I20260807 15:15:14.928025 9204 webserver.cc:460] Webserver started at http://127.31.230.194:37763/ using document root <none> and password file <none>
I20260807 15:15:14.928608 9204 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:14.928701 9204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:14.928946 9204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:14.931051 9204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "11e40373a5dd4252878c99852d074b20"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "fd740c3750d9e4a969d763f07ad99787"
server_key_iv: "01464ba9c8689b4a431ffe4d77c7799f"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.931681 9204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "11e40373a5dd4252878c99852d074b20"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "fd740c3750d9e4a969d763f07ad99787"
server_key_iv: "01464ba9c8689b4a431ffe4d77c7799f"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.935573 9204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:14.938006 9225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.939265 9204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:14.939410 9204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "11e40373a5dd4252878c99852d074b20"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "fd740c3750d9e4a969d763f07ad99787"
server_key_iv: "01464ba9c8689b4a431ffe4d77c7799f"
server_key_version: "encryptionkey@0"
I20260807 15:15:14.939525 9204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:14.964713 9204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:14.967960 9204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:14.968174 9204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:14.968818 9204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:14.969759 9204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:14.969832 9204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.969906 9204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:14.969959 9204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:14.979056 9204 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:44365
I20260807 15:15:14.979079 9387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:44365 every 8 connection(s)
I20260807 15:15:14.979996 9204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:14.982383 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 9204
I20260807 15:15:14.982512 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:14.982884 32667 external_mini_cluster.cc:1428] Setting key d75e261d7af3ce8343fd49da50f3bdad
Aug 07 15:15:14 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:15:14.996028 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.981912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:34573 (local address 127.31.230.254:37793)
0807 15:15:14.982062 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:14.982066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:14.983103 (+ 1037us) server_negotiation.cc:407] Connection header received
0807 15:15:14.984864 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:14.984867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:14.984911 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:14.984982 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:14.986696 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.987298 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.988817 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.989046 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.989134 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:14.992090 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:14.992093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:14.992096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:14.992121 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:14.993860 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:14.994451 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:14.994455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:14.994457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:14.994567 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:14.994928 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:14.994930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:14.994931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:14.995138 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:14.995220 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:14.995757 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 15:15:14.995874 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:15:14.996912 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.982338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:34573)
0807 15:15:14.982910 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:14.982976 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:14.984630 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:14.985171 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:14.985183 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:14.985767 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:14.986500 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:14.986524 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.987435 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.987437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:14.988677 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:14.988692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:14.989219 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:14.989226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:14.989477 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:14.990289 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:14.990306 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:14.991894 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:14.993984 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:14.993990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:14.994008 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:14.994300 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:14.994676 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:14.994679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:14.994680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:14.994829 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:14.995247 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:14.995253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:14.995512 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 15:15:14.995688 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 15:15:14.995971 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":180,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.195@KRBTEST.COM" created.
I20260807 15:15:14.998412 9388 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37793
I20260807 15:15:14.998698 9388 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:14.999332 9388 heartbeater.cc:511] Master 127.31.230.254:37793 requested a full tablet report, sending...
I20260807 15:15:15.000735 8934 ts_manager.cc:194] Registered new tserver with Master: 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365)
I20260807 15:15:15.001412 8934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:34573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab.
Entry for principal oryx/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab.
I20260807 15:15:15.016369 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37793
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:15.134143 9393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:15.134423 9393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:15.134469 9393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:15.138505 9393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:15.138571 9393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:15.138631 9393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:15.144189 9393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:15.145315 9393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:15.146504 9393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:15.156605 9402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.156570 9400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.156570 9404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:15.156620 9393 server_base.cc:1034] running on GCE node
I20260807 15:15:15.157256 9393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:15.157989 9393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:15.159153 9393 hybrid_clock.cc:648] HybridClock initialized: now 1786115715159120 us; error 47 us; skew 500 ppm
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:15.161971 9393 init.cc:377] Logged in from keytab as oryx/127.31.230.195@KRBTEST.COM (short username oryx)
I20260807 15:15:15.162866 9393 webserver.cc:460] Webserver started at http://127.31.230.195:36533/ using document root <none> and password file <none>
I20260807 15:15:15.163460 9393 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:15.163523 9393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.163772 9393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:15.165970 9393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "0bc22d5a2cf8472fa50f7a28c0a75706"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "8feb957b164d62e61cfdbb1c9da83079"
server_key_iv: "fc0070e1dc7794fb9030c09a1b1b91ca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.166545 9393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "0bc22d5a2cf8472fa50f7a28c0a75706"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "8feb957b164d62e61cfdbb1c9da83079"
server_key_iv: "fc0070e1dc7794fb9030c09a1b1b91ca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.170437 9393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:15.172957 9416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.174284 9393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:15.174419 9393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0bc22d5a2cf8472fa50f7a28c0a75706"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "8feb957b164d62e61cfdbb1c9da83079"
server_key_iv: "fc0070e1dc7794fb9030c09a1b1b91ca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.174530 9393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:15.184276 9393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:15.187180 9393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:15.187394 9393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:15.187994 9393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:15.188825 9393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:15.188898 9393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.188970 9393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:15.189013 9393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.198246 9393 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:32959
I20260807 15:15:15.198287 9576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:32959 every 8 connection(s)
I20260807 15:15:15.199146 9393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:15.204138 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 9393
I20260807 15:15:15.204286 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:15.204595 32667 external_mini_cluster.cc:1428] Setting key a5c1bf513c6748cc36d79136b7821a53
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:15.213183 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.200884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:48975 (local address 127.31.230.254:37793)
0807 15:15:15.201030 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.201034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.201962 (+ 928us) server_negotiation.cc:407] Connection header received
0807 15:15:15.203293 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.203295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.203342 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.203421 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.204946 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.205642 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.207208 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.207429 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.207512 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.210042 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.210045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.210047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.210070 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.211391 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.211862 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.211864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.211865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.211940 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.212252 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.212254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.212255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.212426 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.212501 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.212963 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.213061 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:15:15.214094 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.201271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:48975)
0807 15:15:15.201800 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.201843 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.203087 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.203584 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.203591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.204059 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.204767 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.204788 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.205787 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.205791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.207093 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.207103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.207568 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.207570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.207725 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.208364 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.208381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.209892 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.211498 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.211503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.211514 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.211777 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.212022 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.212024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.212025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.212164 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.212516 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.212521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.212757 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.212915 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.213163 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":120,"threads_started":1}
I20260807 15:15:15.215233 9577 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37793
I20260807 15:15:15.215483 9577 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:15.215984 9577 heartbeater.cc:511] Master 127.31.230.254:37793 requested a full tablet report, sending...
I20260807 15:15:15.216995 8934 ts_manager.cc:194] Registered new tserver with Master: 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959)
I20260807 15:15:15.217736 8934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:48975
I20260807 15:15:15.218930 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.230.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.253.keytab.
Entry for principal oryx/127.31.230.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.253.keytab.
I20260807 15:15:15.249025 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:43185
--webserver_interface=127.31.230.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.253.keytab
--principal=oryx/127.31.230.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37793,127.31.230.253:43185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:15.369153 9584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:15.369450 9584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:15.369557 9584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:15.373638 9584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:15.373720 9584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:15.373757 9584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:15.373793 9584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:15.373826 9584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:15.379426 9584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37793,127.31.230.253:43185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.253.keytab
--principal=oryx/127.31.230.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.230.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
W20260807 15:15:15.379706 9584 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.230.254:37793,127.31.230.253:43185'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 15:15:15.380643 9584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:15.381788 9584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:15.390762 9584 server_base.cc:1034] running on GCE node
W20260807 15:15:15.390697 9590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.390724 9589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.390707 9593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:15.391644 9584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:15.392809 9584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:15.394009 9584 hybrid_clock.cc:648] HybridClock initialized: now 1786115715393978 us; error 43 us; skew 500 ppm
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:15.396869 9584 init.cc:377] Logged in from keytab as oryx/127.31.230.253@KRBTEST.COM (short username oryx)
I20260807 15:15:15.397825 9584 webserver.cc:460] Webserver started at http://127.31.230.253:41307/ using document root <none> and password file <none>
I20260807 15:15:15.398403 9584 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:15.398499 9584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.398736 9584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:15.400800 9584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/instance:
uuid: "ee1679942a1e47cbb568531139078826"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "84d43419ff11fbc25ad4b944ae354e82"
server_key_iv: "551f8b0f762bc62cbd8b5125c1661eca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.401372 9584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal/instance:
uuid: "ee1679942a1e47cbb568531139078826"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "84d43419ff11fbc25ad4b944ae354e82"
server_key_iv: "551f8b0f762bc62cbd8b5125c1661eca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.405212 9584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:15:15.407881 9603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.409099 9584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:15.409260 9584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
uuid: "ee1679942a1e47cbb568531139078826"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "84d43419ff11fbc25ad4b944ae354e82"
server_key_iv: "551f8b0f762bc62cbd8b5125c1661eca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.409377 9584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:15.424266 9584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:15.427563 9584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:15.427773 9584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:15.435098 9584 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.253:43185
I20260807 15:15:15.435385 9673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.253:43185 every 8 connection(s)
I20260807 15:15:15.436142 9584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
I20260807 15:15:15.436988 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 9584
I20260807 15:15:15.439368 9674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:15.443358 9674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.253@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:15.452550 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.438585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.31.230.253:43185)
0807 15:15:15.439033 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.439048 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.439083 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:15:15.439804 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.439822 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.440148 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.440501 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.441351 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.442445 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.443326 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.443916 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.444091 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.447191 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.447205 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.447213 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.447240 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.448985 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.449840 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.449851 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.449856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.450052 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.450385 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.450389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.450390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.450853 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.451057 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.451342 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.451643 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":149,"threads_started":1}
I20260807 15:15:15.454465 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.445262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:54270)
0807 15:15:15.445762 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.445817 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.445943 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.446261 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.446267 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.446755 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.446886 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.446892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.447727 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.447765 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.448406 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.448415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.448890 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.448894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.449022 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.449628 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.449648 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.450976 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.452759 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.452767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.452775 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.452990 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.453305 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.453309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.453311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.453415 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.453954 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.453959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.454056 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.454179 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.454317 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":147,"threads_started":1}
I20260807 15:15:15.454541 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.445296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.31.230.254:37793)
0807 15:15:15.445427 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.445431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.445888 (+ 457us) server_negotiation.cc:407] Connection header received
0807 15:15:15.446065 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.446068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.446122 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.446218 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.447016 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.447584 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.448536 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.448774 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.448888 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.451184 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.451190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.451195 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.451227 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.452586 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.453106 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.453109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.453111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.453201 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.453513 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.453516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.453518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.453835 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.453943 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.454274 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.454398 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:15:15.455693 9674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.253@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
I20260807 15:15:15.463104 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.456158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:49554)
0807 15:15:15.456269 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.456285 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.456359 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.456533 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.456535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.456661 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.456755 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.456759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.457837 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.457839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.458333 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.458340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.458737 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.458740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.458810 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.459256 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.459267 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.460402 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.461893 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.461895 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.461897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.462078 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.462312 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.462314 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.462315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.462387 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.462695 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.462697 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.462778 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.462860 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.462951 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 15:15:15.463114 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.456243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.31.230.253:43185)
0807 15:15:15.456359 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.456362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.456374 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:15:15.456406 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.456408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.456439 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.456502 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.456852 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.457732 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.458428 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.458640 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.458714 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.460516 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.460519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.460520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.460539 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.461790 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.462161 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.462163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.462164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.462232 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.462465 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.462467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.462468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.462616 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.462688 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.462919 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.463002 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:15:15.465907 9674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Bootstrap starting.
I20260807 15:15:15.468180 9674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:15.469018 9674 log.cc:824] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:15.471040 9674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: No bootstrap required, opened a new log
I20260807 15:15:15.474094 9674 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } }
I20260807 15:15:15.474326 9674 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:15.474426 9674 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1679942a1e47cbb568531139078826, State: Initialized, Role: FOLLOWER
I20260807 15:15:15.474872 9674 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } }
I20260807 15:15:15.475392 9681 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } } }
I20260807 15:15:15.475508 9681 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:15.476105 9674 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:15:15.481276 9695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:15.481377 9695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:15.484093 9584 master_runner.cc:428] Detected that this master ee1679942a1e47cbb568531139078826 is joining an existing cluster
I20260807 15:15:15.484169 9584 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.230.253:43185
I20260807 15:15:15.496273 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.486643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:49558)
0807 15:15:15.487102 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.487115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.487225 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.487485 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.487488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.487753 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.487924 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.487931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.488958 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.488961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.490218 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.490230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.491646 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.491650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.491725 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.492185 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.492200 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.492739 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.494667 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.494670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.494672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.494961 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.495237 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.495240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.495242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.495328 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.495705 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.495708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.495792 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.495909 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.496063 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":166,"thread_start_us":109,"threads_started":1}
I20260807 15:15:15.496414 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.486643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:54280)
0807 15:15:15.487388 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.487398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.487566 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.487909 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.487913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.488141 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.488249 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.488254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.489028 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.489034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.490219 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.490230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.490894 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.490897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.491009 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.491953 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.491970 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.492535 (+ 565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.494602 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.494608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.494610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.494943 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.495275 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.495278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.495281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.495372 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.495846 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.495849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.495950 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.496051 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.496185 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":237,"spinlock_wait_cycles":9600,"thread_start_us":81,"threads_started":1}
I20260807 15:15:15.496632 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.486803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.31.230.254:37793)
0807 15:15:15.486927 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.486933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.487487 (+ 554us) server_negotiation.cc:407] Connection header received
0807 15:15:15.487686 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.487690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.487774 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.487864 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.488373 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.488910 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.490354 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.490745 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.490869 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.492673 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.492677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.492680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.492721 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.494478 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.495058 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.495061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.495063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.495159 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.495473 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.495476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.495478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.495689 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.495772 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.496112 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.496224 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:15:15.497105 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.486729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.31.230.253:43185)
0807 15:15:15.486934 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.486938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.487198 (+ 260us) server_negotiation.cc:407] Connection header received
0807 15:15:15.487296 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.487299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.487347 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.487440 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.488033 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.488823 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.491063 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.491517 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.491622 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.492874 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.492877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.492880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.492902 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.494541 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.495060 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.495063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.495064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.495139 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.495418 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.495423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.495425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.495609 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.495725 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.495957 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.496975 (+ 1018us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":910,"server-negotiator.queue_time_us":138}
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.254@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
I20260807 15:15:15.511278 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.501373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:49572)
0807 15:15:15.501824 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.501885 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.502047 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.502338 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.502347 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.502791 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.502933 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.502941 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.503605 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.503609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.504596 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.504607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.505023 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.505027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.505147 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.505736 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.505751 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.507458 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.509422 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.509436 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.509445 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.509825 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.510134 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.510139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.510141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.510254 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.510756 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.510762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.510874 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.510992 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.511147 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":94,"threads_started":1}
I20260807 15:15:15.511350 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.501396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.31.230.253:43185)
0807 15:15:15.501521 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.501525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.501985 (+ 460us) server_negotiation.cc:407] Connection header received
0807 15:15:15.502165 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.502167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.502210 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.502278 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.503045 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.503502 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.504717 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.504914 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.504991 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.507607 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.507611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.507613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.507638 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.509298 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.509935 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.509939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.509941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.510029 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.510368 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.510373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.510376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.510634 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.510745 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.511087 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.511203 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:15:15.512976 8934 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.230.253:43185: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ee1679942a1e47cbb568531139078826"
member_type: NON_VOTER
last_known_addr {
host: "127.31.230.253"
port: 43185
}
attrs {
promote: true
}
}
dest_uuid: "1ce63408ebb0481eb74bd908e2211fba"
cas_config_opid_index: -1
I20260807 15:15:15.513988 8934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } }
I20260807 15:15:15.516543 9710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ee1679942a1e47cbb568531139078826 (127.31.230.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
I20260807 15:15:15.517210 9640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:15.518592 9710 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.230.253:43185
I20260807 15:15:15.518817 9640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Refusing update from remote peer 1ce63408ebb0481eb74bd908e2211fba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
WARNING: no policy specified for oryx/127.31.230.252@KRBTEST.COM; defaulting to no policy
I20260807 15:15:15.519099 9584 master.cc:537] Master@127.31.230.253:43185 shutting down...
I20260807 15:15:15.519485 9710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
I20260807 15:15:15.519608 9710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:15.519840 9710 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:15:15.522315 8920 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.252@KRBTEST.COM" created.
W20260807 15:15:15.523975 8920 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba -> Peer ee1679942a1e47cbb568531139078826 (127.31.230.253:43185): Couldn't send request to peer ee1679942a1e47cbb568531139078826. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:15:15.524430 9584 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:15.524761 9584 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:15.524900 9584 tablet_replica.cc:330] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: stopping tablet replica
I20260807 15:15:15.541363 9584 master.cc:552] Master@127.31.230.253:43185 shutdown complete.
I20260807 15:15:15.541440 9584 master_runner.cc:305] Clearing existing system tablet
I20260807 15:15:15.541826 9584 fs_manager.cc:359] Metadata directory not provided
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.252.keytab.
Entry for principal oryx/127.31.230.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.252.keytab.
I20260807 15:15:15.541908 9584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.542892 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:39035
--webserver_interface=127.31.230.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.252.keytab
--principal=oryx/127.31.230.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:15.543316 9584 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.544283 9726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.544488 9584 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:15.544624 9584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.544668 9584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
uuid: "ee1679942a1e47cbb568531139078826"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "84d43419ff11fbc25ad4b944ae354e82"
server_key_iv: "551f8b0f762bc62cbd8b5125c1661eca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.546734 9584 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:15.548691 9584 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:15:15.548815 9584 log.cc:1196] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 15:15:15.549079 9584 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Deleting consensus metadata
I20260807 15:15:15.549273 9584 master_runner.cc:315] Copying system tablet from 127.31.230.254:37793
I20260807 15:15:15.550525 9584 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: tablet copy: Beginning tablet copy session from remote peer at address 127.31.230.254:37793
I20260807 15:15:15.558822 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.551219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.31.230.254:37793)
0807 15:15:15.551379 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.551385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.551580 (+ 195us) server_negotiation.cc:407] Connection header received
0807 15:15:15.551750 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.551753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.551808 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.551898 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.552371 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.553023 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.554004 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.554245 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.554358 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.555540 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.555544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.555547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.555583 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.557067 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.557513 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.557517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.557519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.557649 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.557951 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.557955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.557957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.558151 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.558266 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.558545 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.558661 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:15:15.558838 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.551143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:54286)
0807 15:15:15.551491 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.551503 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.551632 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.551915 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.551918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.552128 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.552250 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.552256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.553169 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.553172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.553875 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.553885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.554366 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.554369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.554492 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.555033 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.555045 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.555381 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.557184 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.557189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.557191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.557405 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.557757 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.557760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.557762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.557854 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.558266 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.558269 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.558374 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.558484 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.558638 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":108,"threads_started":1}
I20260807 15:15:15.559682 8965 tablet_copy_service.cc:140] P 1ce63408ebb0481eb74bd908e2211fba: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ee1679942a1e47cbb568531139078826 ({username='oryx', principal='oryx/127.31.230.253@KRBTEST.COM'} at 127.0.0.1:54286)
I20260807 15:15:15.559872 8965 tablet_copy_service.cc:161] P 1ce63408ebb0481eb74bd908e2211fba: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ee1679942a1e47cbb568531139078826 at {username='oryx', principal='oryx/127.31.230.253@KRBTEST.COM'} at 127.0.0.1:54286: session id = ee1679942a1e47cbb568531139078826-00000000000000000000000000000000
I20260807 15:15:15.561306 8965 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:15:15.562976 9584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:15:15.564921 9584 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: tablet copy: Starting download of 0 data blocks...
I20260807 15:15:15.565145 9584 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: tablet copy: Starting download of 1 WAL segments...
I20260807 15:15:15.566817 9584 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:15:15.568347 8965 tablet_copy_service.cc:342] P 1ce63408ebb0481eb74bd908e2211fba: Request end of tablet copy session ee1679942a1e47cbb568531139078826-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.230.253@KRBTEST.COM'} at 127.0.0.1:54286
I20260807 15:15:15.568468 8965 tablet_copy_service.cc:433] P 1ce63408ebb0481eb74bd908e2211fba: ending tablet copy session ee1679942a1e47cbb568531139078826-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ee1679942a1e47cbb568531139078826
I20260807 15:15:15.576067 9584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:15.577857 9737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.577872 9740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:15.578370 9584 server_base.cc:1034] running on GCE node
W20260807 15:15:15.578374 9738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:15.578670 9584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:15.579003 9584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:15.580133 9584 hybrid_clock.cc:648] HybridClock initialized: now 1786115715580137 us; error 45 us; skew 500 ppm
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:15.582077 9584 init.cc:377] Logged in from keytab as oryx/127.31.230.253@KRBTEST.COM (short username oryx)
I20260807 15:15:15.582572 9584 webserver.cc:460] Webserver started at http://127.31.230.253:38011/ using document root <none> and password file <none>
I20260807 15:15:15.582783 9584 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:15.582839 9584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.584034 9584 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.585023 9751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.585302 9584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.585381 9584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
uuid: "ee1679942a1e47cbb568531139078826"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "84d43419ff11fbc25ad4b944ae354e82"
server_key_iv: "551f8b0f762bc62cbd8b5125c1661eca"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.585489 9584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:15.599709 9584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:15.601824 9584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:15.607003 9584 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.253:43185
I20260807 15:15:15.607661 9584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
I20260807 15:15:15.607904 9820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.253:43185 every 8 connection(s)
I20260807 15:15:15.609208 9821 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:15:15.610100 9821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Bootstrap starting.
I20260807 15:15:15.618327 9821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:15:15.618654 9821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Bootstrap complete.
I20260807 15:15:15.619155 9821 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } }
I20260807 15:15:15.619304 9821 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ee1679942a1e47cbb568531139078826, State: Initialized, Role: LEARNER
I20260807 15:15:15.619421 9821 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } }
I20260807 15:15:15.619793 9824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
I20260807 15:15:15.619951 9824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:15:15.619949 9821 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:15.623468 9840 catalog_manager.cc:1259] Loaded cluster ID: 2ff8203034124349b90f11131fc8c229
I20260807 15:15:15.623544 9840 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: loading cluster ID for follower catalog manager: success
I20260807 15:15:15.625305 9840 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: acquiring CA information for follower catalog manager: success
I20260807 15:15:15.626621 9840 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:15:15.698653 9727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:15.698937 9727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:15.698979 9727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:15.703126 9727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:15.703190 9727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:15.703210 9727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:15.703227 9727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:15.703243 9727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:15.708700 9727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.252.keytab
--principal=oryx/127.31.230.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:39035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.230.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:15.709848 9727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:15.710865 9727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:15.719597 9847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.719614 9849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.719671 9846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:15.720073 9727 server_base.cc:1034] running on GCE node
I20260807 15:15:15.720780 9727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:15.722179 9727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:15.723352 9727 hybrid_clock.cc:648] HybridClock initialized: now 1786115715723320 us; error 45 us; skew 500 ppm
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:15.726467 9727 init.cc:377] Logged in from keytab as oryx/127.31.230.252@KRBTEST.COM (short username oryx)
I20260807 15:15:15.727407 9727 webserver.cc:460] Webserver started at http://127.31.230.252:44777/ using document root <none> and password file <none>
I20260807 15:15:15.727947 9727 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:15.728008 9727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.728171 9727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:15.730329 9727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/instance:
uuid: "425b378830e04c9eb99cc5931824e4d5"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "92b17c4835a64dffb18e751e93b9ad93"
server_key_iv: "12142517c47371c79573d7768babc990"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.730859 9727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal/instance:
uuid: "425b378830e04c9eb99cc5931824e4d5"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "92b17c4835a64dffb18e751e93b9ad93"
server_key_iv: "12142517c47371c79573d7768babc990"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.734790 9727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:15:15.737202 9859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.738518 9727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:15:15.738620 9727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
uuid: "425b378830e04c9eb99cc5931824e4d5"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "92b17c4835a64dffb18e751e93b9ad93"
server_key_iv: "12142517c47371c79573d7768babc990"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.738706 9727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:15.755272 9727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:15.758320 9727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:15.758482 9727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:15.765403 9727 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.252:39035
I20260807 15:15:15.765746 9930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.252:39035 every 8 connection(s)
I20260807 15:15:15.766477 9727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
I20260807 15:15:15.769415 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:15.773341 9933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:15:15.775246 9199 heartbeater.cc:503] Master 127.31.230.254:37793 was elected leader, sending a full tablet report...
I20260807 15:15:15.776381 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 9727
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.252@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.230.252@KRBTEST.COM
I20260807 15:15:15.788867 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.775556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54300 (local address 127.31.230.254:37793)
0807 15:15:15.775709 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.775715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.776906 (+ 1191us) server_negotiation.cc:407] Connection header received
0807 15:15:15.778176 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.778179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.778245 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.778346 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.779397 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.779941 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.780972 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.781201 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.781326 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.784340 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.784344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.784347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.784373 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.786286 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.787044 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.787050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.787053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.787197 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.787609 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.787613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.787616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.787849 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.787979 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.788550 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.788687 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:15:15.789781 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.775936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:54300)
0807 15:15:15.776739 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.776804 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.777947 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.778403 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.778409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.778917 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.779249 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.779265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.780077 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.780080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.780841 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.780850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.781363 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.781368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.781595 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.782309 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.782337 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.784165 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.786403 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.786414 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.786425 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.786909 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.787281 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.787286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.787291 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.787482 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.787989 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.787994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.788254 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.788461 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.788808 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":24,"spinlock_wait_cycles":1280,"thread_start_us":201,"threads_started":1}
I20260807 15:15:15.789781 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.777343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.31.230.252:39035)
0807 15:15:15.777777 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.777797 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.777835 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:15:15.777966 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.777982 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.778229 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.778431 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.779466 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.780423 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.781351 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.781817 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.781956 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.784850 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.784857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.784870 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.784894 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.786936 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.787493 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.787498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.787499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.787628 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.787925 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.787931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.787936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.788228 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.788401 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.788666 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.788863 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":280,"thread_start_us":89,"threads_started":1}
I20260807 15:15:15.791594 9933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.252@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
I20260807 15:15:15.799794 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.792121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:49588)
0807 15:15:15.792262 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.792294 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.792377 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.792783 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.792786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.792956 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.793049 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.793054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.794048 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.794053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.794593 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.794601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.795080 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.795083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.795197 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.795626 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.795636 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.796853 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.798524 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.798526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.798527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.798725 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.798971 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.798973 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.798975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.799050 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.799399 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.799401 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.799478 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.799559 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.799662 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 15:15:15.799837 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.792264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.31.230.253:43185)
0807 15:15:15.792563 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.792567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.792578 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:15.792614 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.792617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.792655 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.792771 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.793171 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.793935 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.794728 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.794964 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.795081 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.796990 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.796993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.796995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.797019 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.798419 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.798813 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.798816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.798817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.798892 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.799127 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.799129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.799130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.799318 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.799394 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.799618 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.799718 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":116,"threads_started":1}
I20260807 15:15:15.800748 9933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.252@KRBTEST.COM for oryx/127.31.230.252@KRBTEST.COM
I20260807 15:15:15.808048 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.801231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.0.0.1:48736)
0807 15:15:15.801347 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.801363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.801439 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.801734 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.801736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.801876 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.801946 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.801951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.802640 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.802642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.803145 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.803152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.803563 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.803566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.803622 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.804088 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.804108 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.805266 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.806820 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.806823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.806824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.807007 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.807245 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.807247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.807248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.807322 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.807637 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.807639 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.807725 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.807806 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.807904 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:15:15.808073 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.801386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.31.230.252:39035)
0807 15:15:15.801496 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.801500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.801510 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:15.801578 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.801581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.801624 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.801689 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.802044 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.802539 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.803241 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.803470 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.803545 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.805391 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.805394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.805396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.805417 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.806712 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.807091 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.807093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.807095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.807164 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.807399 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.807401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.807402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.807557 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.807631 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.807866 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.807954 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:15:15.811539 9933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Bootstrap starting.
I20260807 15:15:15.813999 9933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:15.814766 9933 log.cc:824] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:15.816727 9933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: No bootstrap required, opened a new log
I20260807 15:15:15.819753 9933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } }
I20260807 15:15:15.820019 9933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:15.820102 9933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425b378830e04c9eb99cc5931824e4d5, State: Initialized, Role: FOLLOWER
I20260807 15:15:15.820603 9933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } }
I20260807 15:15:15.821188 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } } }
I20260807 15:15:15.821363 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:15.822180 9933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:15:15.827018 9961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:15.827111 9961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:15.830585 9727 master_runner.cc:428] Detected that this master 425b378830e04c9eb99cc5931824e4d5 is joining an existing cluster
I20260807 15:15:15.830664 9727 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.230.252:39035
I20260807 15:15:15.843146 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 9018
I20260807 15:15:15.870033 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:39951
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:37793
--webserver_port=34851
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:15.876346 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.832913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.31.230.254:37793)
0807 15:15:15.833053 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.833057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.833291 (+ 234us) server_negotiation.cc:407] Connection header received
0807 15:15:15.833455 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.833460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.833520 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.833603 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.834329 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.835193 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.836582 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.836920 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.836985 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.871555 (+ 34570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.871560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.871565 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.871605 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.874242 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.874782 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.874787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.874792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.874881 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875216 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.875219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.875221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.875492 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.875573 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.876014 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.876158 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:15:15.877283 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.832782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:54316)
0807 15:15:15.833213 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.833228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.833336 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.833772 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.833774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.834025 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.834191 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.834201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.835329 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.835332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.836440 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.836454 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.837123 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.837127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.837390 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.870533 (+ 33143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.870555 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.871068 (+ 513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.874402 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.874405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.874407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.874645 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.875008 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.875014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.875109 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.875655 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.875657 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.875806 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.876886 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.877057 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":165,"spinlock_wait_cycles":3328,"thread_start_us":114,"threads_started":1}
I20260807 15:15:15.877571 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.833560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.31.230.253:43185)
0807 15:15:15.833694 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.833701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.834257 (+ 556us) server_negotiation.cc:407] Connection header received
0807 15:15:15.834426 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.834429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.834477 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.834528 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.835196 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.836046 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.837300 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.837693 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.837787 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.872847 (+ 35060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.872853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.872857 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.872894 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.874618 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875113 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.875117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.875118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.875223 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875999 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.876003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.876005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.876239 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.876316 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.877247 (+ 931us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.877407 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:15:15.878230 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.833094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.31.230.252:39035)
0807 15:15:15.833195 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.833199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.833935 (+ 736us) server_negotiation.cc:407] Connection header received
0807 15:15:15.834030 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.834032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.834076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.834147 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.834665 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.835365 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.836567 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.836878 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.836959 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.870831 (+ 33872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.870836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.870839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.870871 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.872662 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875632 (+ 2970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.875637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.875640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.875714 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.876656 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.876660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.876662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.876852 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.876934 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.877978 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.878094 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:15:15.878777 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.833446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:49590)
0807 15:15:15.834176 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.834189 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.834321 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.834671 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.834674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.834929 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.835055 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.835063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.836207 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.836210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.837153 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.837165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.837939 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.837942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.838033 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.838440 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.838455 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.871714 (+ 33259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.874747 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.874753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.874758 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.875000 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.875347 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.875352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.875446 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.876400 (+ 954us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.876404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.876499 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.877769 (+ 1270us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.878600 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":2,"thread_start_us":134,"threads_started":1}
I20260807 15:15:15.880105 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.833052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.0.0.1:48750)
0807 15:15:15.833825 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.833839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.833942 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.834243 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.834248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.834447 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.834553 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.834559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.835473 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.835476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.836460 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.836466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.837123 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.837127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.837387 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.838065 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.838077 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.870366 (+ 32289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.873236 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.873243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.873271 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.875542 (+ 2271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.875811 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.875814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.875816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.875900 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.876975 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.876978 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.877056 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.879795 (+ 2739us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.879936 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":203,"spinlock_wait_cycles":7296,"thread_start_us":90,"threads_started":1}
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115714, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.254@KRBTEST.COM for oryx/127.31.230.252@KRBTEST.COM
I20260807 15:15:15.890471 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.882267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.0.0.1:48764)
0807 15:15:15.882443 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.882457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.882574 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.882881 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.882884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.883080 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.883168 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.883174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.883924 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.883928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.884493 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.884501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.884975 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.884979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.885092 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.885597 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.885619 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.887086 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.888813 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.888816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.888818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.889022 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.889313 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.889317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.889318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.889409 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.889912 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.889915 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.890026 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.890139 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.890309 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 15:15:15.890470 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.882307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.31.230.252:39035)
0807 15:15:15.882410 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.882414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.882532 (+ 118us) server_negotiation.cc:407] Connection header received
0807 15:15:15.882721 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.882723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.882767 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.882861 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.883296 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.883808 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.884616 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.884854 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.884970 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.887229 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.887232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.887235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.887259 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.888672 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.889124 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.889128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.889130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.889213 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.889505 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.889510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.889511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.889732 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.889922 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.890206 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.890315 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:15:15.891885 8934 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.230.252:39035: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "425b378830e04c9eb99cc5931824e4d5"
member_type: NON_VOTER
last_known_addr {
host: "127.31.230.252"
port: 39035
}
attrs {
promote: true
}
}
dest_uuid: "1ce63408ebb0481eb74bd908e2211fba"
cas_config_opid_index: 5
I20260807 15:15:15.892518 8934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } }
I20260807 15:15:15.893980 9710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.894145 9895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:15.895496 9710 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.230.252:39035
I20260807 15:15:15.895613 9895 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 FOLLOWER]: Refusing update from remote peer 1ce63408ebb0481eb74bd908e2211fba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:15:15.895668 9710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.895757 9710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:15.896499 9711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:15.897650 9727 master.cc:537] Master@127.31.230.252:39035 shutting down...
I20260807 15:15:15.901508 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.896106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:49596)
0807 15:15:15.896654 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.896668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.896757 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.897217 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.897221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.897239 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:15.897344 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.897351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.898455 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.898460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.899702 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.899714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.901092 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.901095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.901187 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.901243 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.901320 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.901390 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482}
I20260807 15:15:15.901947 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.893868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49596 (local address 127.31.230.253:43185)
0807 15:15:15.896564 (+ 2696us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.896569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.896847 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:15:15.896905 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.896908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.896975 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.897071 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:15.897482 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.898307 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.899851 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.900744 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.900844 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.900985 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.901734 (+ 749us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.901834 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2596}
I20260807 15:15:15.902375 9789 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 LEARNER]: Refusing update from remote peer 1ce63408ebb0481eb74bd908e2211fba: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:15:15.902892 9711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:15:15.905391 9789 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.907531 9824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ce63408ebb0481eb74bd908e2211fba. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.907660 9824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:15:15.907778 9824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.907866 9824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:15:15.909966 9710 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: attempting to promote NON_VOTER ee1679942a1e47cbb568531139078826 to VOTER
I20260807 15:15:15.910401 9710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } }
I20260807 15:15:15.911105 9895 raft_consensus.cc:858] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } }
W20260807 15:15:15.911199 9895 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } }
I20260807 15:15:15.912689 9895 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ee1679942a1e47cbb568531139078826 (127.31.230.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
W20260807 15:15:15.913199 8918 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba -> Peer 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252:39035): Couldn't send request to peer 425b378830e04c9eb99cc5931824e4d5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:15:15.913553 9789 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 LEARNER]: Refusing update from remote peer 1ce63408ebb0481eb74bd908e2211fba: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 15:15:15.914002 9711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:15:15.916795 9789 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ee1679942a1e47cbb568531139078826 (127.31.230.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.917088 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ce63408ebb0481eb74bd908e2211fba. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
I20260807 15:15:15.917199 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 15:15:15.917299 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
I20260807 15:15:15.917362 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 15:15:15.918471 9824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.918588 9824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:15.919081 9979 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ee1679942a1e47cbb568531139078826 (127.31.230.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.919462 9727 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 15:15:15.919725 9727 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 15:15:15.920656 9981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: NON_VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: true } } }
I20260807 15:15:15.920818 9981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 15:15:15.921101 9727 tablet_replica.cc:330] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: stopping tablet replica
I20260807 15:15:15.922003 9710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.922132 9710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:15.922446 9710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:15.922539 9710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:15.942047 9727 master.cc:552] Master@127.31.230.252:39035 shutdown complete.
I20260807 15:15:15.942129 9727 master_runner.cc:305] Clearing existing system tablet
I20260807 15:15:15.942389 9727 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:15.942469 9727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.943825 9727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:15.944821 9990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.945111 9727 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:15.945276 9727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.945315 9727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
uuid: "425b378830e04c9eb99cc5931824e4d5"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "92b17c4835a64dffb18e751e93b9ad93"
server_key_iv: "12142517c47371c79573d7768babc990"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.947111 9727 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:15.949007 9727 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:15:15.949118 9727 log.cc:1196] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 15:15:15.949419 9727 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Deleting consensus metadata
I20260807 15:15:15.949690 9727 master_runner.cc:315] Copying system tablet from 127.31.230.254:37793
I20260807 15:15:15.950863 9727 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: tablet copy: Beginning tablet copy session from remote peer at address 127.31.230.254:37793
I20260807 15:15:15.959273 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.951419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:54332)
0807 15:15:15.951751 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.951764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:15.951900 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:15.952194 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:15.952196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:15.952408 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:15.952493 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.952499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.953295 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.953300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.954032 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:15.954042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.954532 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.954536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:15.954638 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.955197 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:15.955211 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:15.955551 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:15.957485 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.957490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.957493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.957855 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.958154 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:15.958157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:15.958159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:15.958255 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:15.958673 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:15.958677 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:15.958795 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 15:15:15.958909 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:15:15.959076 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":95,"threads_started":1}
I20260807 15:15:15.959259 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.951502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.31.230.254:37793)
0807 15:15:15.951648 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:15.951654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:15.951851 (+ 197us) server_negotiation.cc:407] Connection header received
0807 15:15:15.952024 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:15.952027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:15.952086 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:15.952175 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:15.952620 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.953167 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.954160 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:15.954405 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:15.954524 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:15.955707 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:15.955710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:15.955713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:15.955740 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:15.957361 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.957945 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.957949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.957951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.958044 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:15.958358 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:15.958361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:15.958363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:15.958566 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:15.958682 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:15.958968 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 15:15:15.959083 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:15:15.960093 8965 tablet_copy_service.cc:140] P 1ce63408ebb0481eb74bd908e2211fba: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 425b378830e04c9eb99cc5931824e4d5 ({username='oryx', principal='oryx/127.31.230.252@KRBTEST.COM'} at 127.0.0.1:54332)
I20260807 15:15:15.960218 8965 tablet_copy_service.cc:161] P 1ce63408ebb0481eb74bd908e2211fba: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 425b378830e04c9eb99cc5931824e4d5 at {username='oryx', principal='oryx/127.31.230.252@KRBTEST.COM'} at 127.0.0.1:54332: session id = 425b378830e04c9eb99cc5931824e4d5-00000000000000000000000000000000
I20260807 15:15:15.961117 8965 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:15:15.962441 9727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:15:15.964471 9727 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: tablet copy: Starting download of 0 data blocks...
I20260807 15:15:15.964697 9727 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: tablet copy: Starting download of 1 WAL segments...
I20260807 15:15:15.966248 9727 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:15:15.967924 8965 tablet_copy_service.cc:342] P 1ce63408ebb0481eb74bd908e2211fba: Request end of tablet copy session 425b378830e04c9eb99cc5931824e4d5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.230.252@KRBTEST.COM'} at 127.0.0.1:54332
I20260807 15:15:15.968027 8965 tablet_copy_service.cc:433] P 1ce63408ebb0481eb74bd908e2211fba: ending tablet copy session 425b378830e04c9eb99cc5931824e4d5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 425b378830e04c9eb99cc5931824e4d5
I20260807 15:15:15.975070 9727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:15.976792 9999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.976805 10000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:15.986303 10002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:15.986901 9727 server_base.cc:1034] running on GCE node
I20260807 15:15:15.987185 9727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:15.987591 9727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:15.988765 9727 hybrid_clock.cc:648] HybridClock initialized: now 1786115715988742 us; error 44 us; skew 500 ppm
Aug 07 15:15:15 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115715, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:15.990620 9727 init.cc:377] Logged in from keytab as oryx/127.31.230.252@KRBTEST.COM (short username oryx)
I20260807 15:15:15.991226 9727 webserver.cc:460] Webserver started at http://127.31.230.252:34717/ using document root <none> and password file <none>
I20260807 15:15:15.991492 9727 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:15.991557 9727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:15.993120 9727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.994292 10009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:15.994565 9727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:15.994674 9727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
uuid: "425b378830e04c9eb99cc5931824e4d5"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "92b17c4835a64dffb18e751e93b9ad93"
server_key_iv: "12142517c47371c79573d7768babc990"
server_key_version: "encryptionkey@0"
I20260807 15:15:15.994791 9727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:16.004369 9388 heartbeater.cc:503] Master 127.31.230.254:37793 was elected leader, sending a full tablet report...
I20260807 15:15:16.012460 9727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:16.014973 9727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:16.020655 9727 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.252:39035
I20260807 15:15:16.020676 10085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.252:39035 every 8 connection(s)
I20260807 15:15:16.021646 9727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
I20260807 15:15:16.023257 10086 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:15:16.024196 10086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Bootstrap starting.
I20260807 15:15:16.032496 10086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:15:16.032860 10086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: Bootstrap complete.
I20260807 15:15:16.033504 10086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } }
I20260807 15:15:16.033718 10086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 425b378830e04c9eb99cc5931824e4d5, State: Initialized, Role: LEARNER
I20260807 15:15:16.033864 10086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } }
I20260807 15:15:16.034250 10093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:16.034441 10086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:16.034441 10093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: LEARNER
W20260807 15:15:16.036559 9975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:16.036928 9975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:16.037006 9975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 15:15:16.038980 10113 catalog_manager.cc:1259] Loaded cluster ID: 2ff8203034124349b90f11131fc8c229
I20260807 15:15:16.039067 10113 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: loading cluster ID for follower catalog manager: success
I20260807 15:15:16.040745 10113 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: acquiring CA information for follower catalog manager: success
I20260807 15:15:16.042119 10113 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:15:16.043593 9975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:16.043668 9975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:16.043767 9975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:16.051862 9975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:39951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=34851
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:16.053246 9975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:16.054581 9975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:16.064965 10116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:16.065083 9975 server_base.cc:1034] running on GCE node
W20260807 15:15:16.065003 10117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:16.065011 10119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:16.065743 9975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:16.066411 9975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:16.067605 9975 hybrid_clock.cc:648] HybridClock initialized: now 1786115716067587 us; error 28 us; skew 500 ppm
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:16.070832 9975 init.cc:377] Logged in from keytab as oryx/127.31.230.193@KRBTEST.COM (short username oryx)
I20260807 15:15:16.071722 9975 webserver.cc:460] Webserver started at http://127.31.230.193:34851/ using document root <none> and password file <none>
I20260807 15:15:16.072283 9975 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:16.072362 9975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:16.076444 9975 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:16.078738 10129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.080221 9975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:16.080325 9975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "ded878401a1f46989497aec764dc1942"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "c204e2f5ecc9d56dc91aac1b115db771"
server_key_iv: "09e12ab25fc7729277d360fdaa93f428"
server_key_version: "encryptionkey@0"
I20260807 15:15:16.080902 9975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:16.094798 9975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:16.098521 9975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:16.098687 9975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:16.099262 9975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:16.100158 9975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:16.100216 9975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.100260 9975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:16.100282 9975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.109454 9975 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:39951
I20260807 15:15:16.109560 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:39951 every 8 connection(s)
I20260807 15:15:16.110586 9975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:16.118834 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 9975
I20260807 15:15:16.119059 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 9204
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
I20260807 15:15:16.133713 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.112436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:41665 (local address 127.31.230.254:37793)
0807 15:15:16.112573 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.112578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.113776 (+ 1198us) server_negotiation.cc:407] Connection header received
0807 15:15:16.115167 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.115170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.115261 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.115360 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.116926 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.117780 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.122895 (+ 5115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.123402 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.123527 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.127571 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.127575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.127578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.127609 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.130745 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.131616 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.131619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.131620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.131700 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.132187 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.132194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.132198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.132456 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.132555 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.133337 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.133484 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:15:16.134784 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.112918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:40147 (local address 127.31.230.253:43185)
0807 15:15:16.113053 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.113057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.114009 (+ 952us) server_negotiation.cc:407] Connection header received
0807 15:15:16.115301 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.115305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.115370 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.115469 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.116910 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.117452 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.126741 (+ 9289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.127101 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.127195 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.130565 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.130569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.130573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.130611 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.132667 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.133137 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.133140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.133142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.133237 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.133593 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.133597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.133599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.133826 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.133915 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.134523 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.134642 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:15:16.135543 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.112977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.31.230.193:40147)
0807 15:15:16.113921 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.113934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.114921 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.115561 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.115570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.116157 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.116739 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.116758 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.117656 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.117661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.126564 (+ 8903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.126579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.127553 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.127556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.127668 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.128721 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.128734 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.130365 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.132780 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.132783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.132785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.133014 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.133364 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.133367 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.133369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.133457 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.134220 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.134223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.134325 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.135191 (+ 866us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.135354 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"mutex_wait_us":56,"spinlock_wait_cycles":42112,"thread_start_us":193,"threads_started":1}
I20260807 15:15:16.135543 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.112977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:41665)
0807 15:15:16.113600 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.113655 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.114931 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.115559 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.115566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.116242 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.116761 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.116772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.117899 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.117902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.122711 (+ 4809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.122726 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.123855 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.123864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.124133 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.125293 (+ 1160us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.125318 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.127360 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.130954 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.130960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.130980 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.131341 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.131813 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.131816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.131818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.132051 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.132636 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.132644 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.133017 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.133924 (+ 907us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.134363 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":193,"thread_start_us":754,"threads_started":1}
I20260807 15:15:16.137287 10295 heartbeater.cc:347] Connected to a master server at 127.31.230.253:43185
I20260807 15:15:16.137625 10295 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.138262 10295 heartbeater.cc:511] Master 127.31.230.253:43185 requested a full tablet report, sending...
I20260807 15:15:16.139081 10294 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37793
I20260807 15:15:16.139174 10294 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.139353 10294 heartbeater.cc:511] Master 127.31.230.254:37793 requested a full tablet report, sending...
I20260807 15:15:16.140056 9779 ts_manager.cc:194] Registered new tserver with Master: ded878401a1f46989497aec764dc1942 (127.31.230.193:39951)
I20260807 15:15:16.140259 8934 ts_manager.cc:194] Re-registered known tserver with Master: ded878401a1f46989497aec764dc1942 (127.31.230.193:39951)
I20260807 15:15:16.141165 8934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:41665
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.252@KRBTEST.COM
I20260807 15:15:16.145874 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:44365
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:37793
--webserver_port=37763
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:16.148794 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.112435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46247 (local address 127.31.230.252:39035)
0807 15:15:16.112693 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.112707 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.114268 (+ 1561us) server_negotiation.cc:407] Connection header received
0807 15:15:16.115612 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.115615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.115688 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.115811 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.116925 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.118001 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.139732 (+ 21731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.140149 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.140247 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.144901 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.144905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.144907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.144988 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.146884 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.147450 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.147454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.147456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.147564 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.147880 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.147883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.147885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.148092 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.148206 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.148492 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.148631 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260807 15:15:16.148809 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.112977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.31.230.193:46247)
0807 15:15:16.114154 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.114168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.114921 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.115847 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.115850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.116215 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.116764 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.116772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.118114 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.118119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.139553 (+ 21434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.139564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.140454 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.140460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.140581 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.141195 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.141208 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.144715 (+ 3507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.147046 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.147049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.147052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.147327 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.147663 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.147666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.147668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.147761 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.148195 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.148198 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.148312 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.148417 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.148599 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":289,"spinlock_wait_cycles":20608,"thread_start_us":159,"threads_started":1}
I20260807 15:15:16.150079 10292 heartbeater.cc:347] Connected to a master server at 127.31.230.252:39035
I20260807 15:15:16.150218 10292 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.150473 10292 heartbeater.cc:511] Master 127.31.230.252:39035 requested a full tablet report, sending...
I20260807 15:15:16.151678 10038 ts_manager.cc:194] Registered new tserver with Master: ded878401a1f46989497aec764dc1942 (127.31.230.193:39951)
I20260807 15:15:16.220283 9577 heartbeater.cc:503] Master 127.31.230.254:37793 was elected leader, sending a full tablet report...
W20260807 15:15:16.265573 10304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:16.265856 10304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:16.265902 10304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:16.270062 10304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:16.270159 10304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:16.270260 10304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:16.277010 10304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:44365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=37763
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:16.278362 10304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:16.279899 10304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:16.293448 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:16.293516 10311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:16.293442 10314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:16.293970 10304 server_base.cc:1034] running on GCE node
I20260807 15:15:16.295189 10304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:16.296248 10304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:16.297493 10304 hybrid_clock.cc:648] HybridClock initialized: now 1786115716297408 us; error 98 us; skew 500 ppm
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:16.301894 10304 init.cc:377] Logged in from keytab as oryx/127.31.230.194@KRBTEST.COM (short username oryx)
I20260807 15:15:16.302983 10304 webserver.cc:460] Webserver started at http://127.31.230.194:37763/ using document root <none> and password file <none>
I20260807 15:15:16.303800 10304 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:16.303915 10304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:16.308251 10304 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:16.310408 10323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.311703 10304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:16.311843 10304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "11e40373a5dd4252878c99852d074b20"
format_stamp: "Formatted at 2026-08-07 15:15:14 on dist-test-slave-gkw7"
server_key: "fd740c3750d9e4a969d763f07ad99787"
server_key_iv: "01464ba9c8689b4a431ffe4d77c7799f"
server_key_version: "encryptionkey@0"
I20260807 15:15:16.312398 10304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:16.326048 10304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:16.329198 10304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:16.329411 10304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:16.330091 10304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:16.330999 10304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:16.331049 10304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.331089 10304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:16.331147 10304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.341401 10304 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:44365
I20260807 15:15:16.341454 10483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:44365 every 8 connection(s)
I20260807 15:15:16.342543 10304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:16.345077 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 10304
I20260807 15:15:16.345266 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 9393
I20260807 15:15:16.357100 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:32959
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:37793
--webserver_port=36533
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.252@KRBTEST.COM
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:16.365031 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.344489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:47869 (local address 127.31.230.253:43185)
0807 15:15:16.344685 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.344692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.349347 (+ 4655us) server_negotiation.cc:407] Connection header received
0807 15:15:16.350072 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.350079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.350148 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.350209 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.352503 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.353385 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.355564 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.356017 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.356118 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.359847 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.359851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.359854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.359887 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.361945 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.362872 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.362879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.362883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.363005 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.363514 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.363535 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.363539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.363811 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.363907 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.364682 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.364841 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:15:16.365360 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.345245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:33947 (local address 127.31.230.252:39035)
0807 15:15:16.345436 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.345441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.348871 (+ 3430us) server_negotiation.cc:407] Connection header received
0807 15:15:16.350085 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.350089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.350150 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.350212 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.352522 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.353377 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.355507 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.355890 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.355979 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.360872 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.360879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.360884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.360926 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.363039 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.363639 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.363644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.363648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.363767 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.364145 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.364150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.364154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.364426 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.364527 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.364964 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.365136 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 15:15:16.366397 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.344669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:58857 (local address 127.31.230.254:37793)
0807 15:15:16.344816 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.344822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.348799 (+ 3977us) server_negotiation.cc:407] Connection header received
0807 15:15:16.350082 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.350087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.350150 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.350212 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.352530 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.353377 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.355555 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.355894 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.355979 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.361519 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.361524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.361556 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.361595 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.363768 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.364358 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.364362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.364364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.364466 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.364827 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.364831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.364832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.365079 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.365178 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.366110 (+ 932us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.366247 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:15:16.366931 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.345104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.31.230.194:33947)
0807 15:15:16.348775 (+ 3671us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.348790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.349798 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.350656 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.350666 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.351405 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.352311 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.352334 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.353623 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.353627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.355332 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.355349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.356227 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.356232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.356673 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.357835 (+ 1162us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.357854 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.360601 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.363191 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.363197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.363202 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.363502 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.363906 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.363911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.363915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.364022 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.364602 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.364605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.364722 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.365615 (+ 893us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.365826 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3592,"mutex_wait_us":415,"spinlock_wait_cycles":184832,"thread_start_us":189,"threads_started":1}
I20260807 15:15:16.366931 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.344999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.31.230.194:47869)
0807 15:15:16.349232 (+ 4233us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.349248 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.349811 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.350657 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.350668 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.351391 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.352311 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.352334 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.353625 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.353631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.355410 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.355425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.356311 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.356317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.356637 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.357463 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.357490 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.359562 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.362067 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.362076 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.362097 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.362717 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.363156 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.363163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.363165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.363372 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.363975 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.363983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.364347 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.365243 (+ 896us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.365708 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4141,"mutex_wait_us":931,"spinlock_wait_cycles":15616,"thread_start_us":152,"threads_started":1}
I20260807 15:15:16.368429 10484 heartbeater.cc:347] Connected to a master server at 127.31.230.252:39035
I20260807 15:15:16.368407 10486 heartbeater.cc:347] Connected to a master server at 127.31.230.253:43185
I20260807 15:15:16.368775 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.344999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:58857)
0807 15:15:16.348596 (+ 3597us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.348662 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.349798 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.350656 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.350666 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.351411 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.352311 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.352334 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.353623 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.353627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.355401 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.355412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.356227 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.356232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.356693 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.357716 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.357735 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.361306 (+ 3571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.363935 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.363939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.363942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.364216 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.364600 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.364606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.364610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.364708 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.365783 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.365786 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.365879 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.366827 (+ 948us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.366991 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3337,"mutex_wait_us":598,"spinlock_wait_cycles":52992,"thread_start_us":173,"threads_started":1}
I20260807 15:15:16.368727 10484 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.368727 10486 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.369700 10484 heartbeater.cc:511] Master 127.31.230.252:39035 requested a full tablet report, sending...
I20260807 15:15:16.369753 10485 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37793
I20260807 15:15:16.369794 10486 heartbeater.cc:511] Master 127.31.230.253:43185 requested a full tablet report, sending...
I20260807 15:15:16.369848 10485 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.370016 10485 heartbeater.cc:511] Master 127.31.230.254:37793 requested a full tablet report, sending...
I20260807 15:15:16.370952 10038 ts_manager.cc:194] Registered new tserver with Master: 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365)
I20260807 15:15:16.370959 9779 ts_manager.cc:194] Registered new tserver with Master: 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365)
I20260807 15:15:16.371415 8934 ts_manager.cc:194] Re-registered known tserver with Master: 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365)
I20260807 15:15:16.372093 8934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:58857
W20260807 15:15:16.482210 10492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:16.482537 10492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:16.482625 10492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:16.486999 10492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:16.487093 10492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:16.487185 10492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:16.492933 10492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:32959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=36533
--tserver_master_addrs=127.31.230.254:37793,127.31.230.253:43185,127.31.230.252:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:16.494127 10492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:16.495276 10492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:16.506086 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.500977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.31.230.252:39035)
0807 15:15:16.501201 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.501207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.501468 (+ 261us) server_negotiation.cc:407] Connection header received
0807 15:15:16.501691 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.501697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.501799 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.501910 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:16.502244 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.502792 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.504388 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.505009 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.505175 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.505352 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.505851 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.505949 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 15:15:16.506460 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.500865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.0.0.1:58570)
0807 15:15:16.501316 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.501336 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.501500 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.501944 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.501948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.501973 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:16.502108 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.502115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.502905 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.502908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.504235 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.504248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.505416 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.505419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.505497 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.505516 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.505821 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.505991 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":70,"threads_started":1}
W20260807 15:15:16.506448 10499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:16.506420 10504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:16.506419 10506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:16.507010 10492 server_base.cc:1034] running on GCE node
I20260807 15:15:16.507129 10049 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 15:15:16.507402 10093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ce63408ebb0481eb74bd908e2211fba. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:16.507500 10492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:16.507561 10093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:15:16.508260 10492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:16.509445 10492 hybrid_clock.cc:648] HybridClock initialized: now 1786115716509423 us; error 29 us; skew 500 ppm
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:16.512356 10492 init.cc:377] Logged in from keytab as oryx/127.31.230.195@KRBTEST.COM (short username oryx)
I20260807 15:15:16.513216 10492 webserver.cc:460] Webserver started at http://127.31.230.195:36533/ using document root <none> and password file <none>
I20260807 15:15:16.513844 10492 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:16.513926 10492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:16.518165 10492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:15:16.520361 10517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.521880 10492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:16.522019 10492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0bc22d5a2cf8472fa50f7a28c0a75706"
format_stamp: "Formatted at 2026-08-07 15:15:15 on dist-test-slave-gkw7"
server_key: "8feb957b164d62e61cfdbb1c9da83079"
server_key_iv: "fc0070e1dc7794fb9030c09a1b1b91ca"
server_key_version: "encryptionkey@0"
I20260807 15:15:16.522560 10492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:16.565155 10492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:16.568576 10492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:16.568806 10492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:16.569484 10492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:16.570492 10492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:16.570574 10492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.570668 10492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:16.570731 10492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:16.581185 10492 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:32959
I20260807 15:15:16.581210 10682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:32959 every 8 connection(s)
I20260807 15:15:16.582337 10492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:16.586305 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 10492
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.253@KRBTEST.COM
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
Aug 07 15:15:16 dist-test-slave-gkw7 krb5kdc[8873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115716, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.252@KRBTEST.COM
I20260807 15:15:16.603037 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.584306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:43003 (local address 127.31.230.252:39035)
0807 15:15:16.584498 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.584505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.585450 (+ 945us) server_negotiation.cc:407] Connection header received
0807 15:15:16.586862 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.586866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.586931 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.586997 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.589002 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.589780 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.591700 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.592067 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.592175 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.598683 (+ 6508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.598688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.598693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.598720 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.600575 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.601391 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.601398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.601400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.601498 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.601910 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.601914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.601916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.602163 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.602246 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.602707 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.602844 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:15:16.603834 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.584296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:50611 (local address 127.31.230.254:37793)
0807 15:15:16.584438 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.584443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.585444 (+ 1001us) server_negotiation.cc:407] Connection header received
0807 15:15:16.586783 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.586786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.586850 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.586928 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.589878 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.590635 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.591751 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.592098 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.592180 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.596339 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.596343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.596347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.596378 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.598583 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.600964 (+ 2381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.600970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.600972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.601073 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.601594 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.601599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.601602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.601849 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.601956 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.603545 (+ 1589us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.603674 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:15:16.604192 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.584754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.31.230.195:43003)
0807 15:15:16.585288 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.585343 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.586541 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.587230 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.587238 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.587873 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.588832 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.588848 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.589950 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.589953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.591526 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.591541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.592386 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.592391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.594833 (+ 2442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.596041 (+ 1208us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.596715 (+ 674us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.598503 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.600989 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.600992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.600994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.601250 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.601660 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.601665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.601670 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.601754 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.602324 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.602327 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.602424 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.603272 (+ 848us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.603397 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":1138,"spinlock_wait_cycles":1485824,"thread_start_us":158,"threads_started":1}
I20260807 15:15:16.604483 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.584754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:50611)
0807 15:15:16.585289 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.585341 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.586533 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.587227 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.587234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.587865 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.589223 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.589234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.590808 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.590811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.591588 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.591599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.592332 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.592336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.592974 (+ 638us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.594170 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.594185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.596137 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.598725 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.598733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.598751 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.600841 (+ 2090us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.601196 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.601204 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.601209 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.601434 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.602018 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.602025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.602390 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.602614 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.602998 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":506,"spinlock_wait_cycles":64512,"thread_start_us":139,"threads_started":1}
I20260807 15:15:16.605099 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.589127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58580 (local address 127.31.230.252:39035)
0807 15:15:16.592587 (+ 3460us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.592591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.592605 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:16.592639 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.592641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.592685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.592798 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.593645 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.594390 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.595789 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.596165 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.596282 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.599425 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.599429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.599431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.599460 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.602083 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.602562 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.602569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.602574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.602662 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.602984 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.602988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.602990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.603605 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.603736 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.604803 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.604930 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3386,"thread_start_us":76,"threads_started":1}
I20260807 15:15:16.605419 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.586068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:53271 (local address 127.31.230.253:43185)
0807 15:15:16.587643 (+ 1575us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.587650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.587669 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:15:16.587714 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.587717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.587773 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.587859 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.588948 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.589777 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.591696 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.592025 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.592134 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.596482 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.596486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.596488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.596516 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.601444 (+ 4928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.602042 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.602045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.602047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.602134 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.602486 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.602491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.602492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.602737 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.602837 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.604317 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.605260 (+ 943us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472}
I20260807 15:15:16.605608 10683 heartbeater.cc:347] Connected to a master server at 127.31.230.252:39035
I20260807 15:15:16.605608 10684 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37793
I20260807 15:15:16.605890 10684 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.605890 10683 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.604206 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.585889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.31.230.195:53271)
0807 15:15:16.586224 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.586237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.586533 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.587989 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.587993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.588159 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:16.588762 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.588784 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.589944 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.589947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.591517 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.591528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.592332 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.592336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.592605 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.593269 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:16.593294 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:16.596348 (+ 3054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:16.601608 (+ 5260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.601615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.601620 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.601909 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.602261 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:16.602267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:16.602272 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:16.602366 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:16.602886 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:16.602889 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:16.602979 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.603108 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.603262 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":276,"thread_start_us":102,"threads_started":1}
I20260807 15:15:16.606568 10684 heartbeater.cc:511] Master 127.31.230.254:37793 requested a full tablet report, sending...
I20260807 15:15:16.606566 10683 heartbeater.cc:511] Master 127.31.230.252:39035 requested a full tablet report, sending...
I20260807 15:15:16.607141 10685 heartbeater.cc:347] Connected to a master server at 127.31.230.253:43185
I20260807 15:15:16.607218 10685 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:16.607390 10685 heartbeater.cc:511] Master 127.31.230.253:43185 requested a full tablet report, sending...
I20260807 15:15:16.607801 8934 ts_manager.cc:194] Re-registered known tserver with Master: 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959)
I20260807 15:15:16.608073 9779 ts_manager.cc:194] Registered new tserver with Master: 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959)
I20260807 15:15:16.608528 8934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:50611
I20260807 15:15:16.609442 10037 ts_manager.cc:194] Registered new tserver with Master: 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959)
I20260807 15:15:16.611042 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.588611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.31.230.254:37793)
0807 15:15:16.589458 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.589474 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.590471 (+ 997us) server_negotiation.cc:407] Connection header received
0807 15:15:16.590699 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.590702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.590765 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.590885 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.593636 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.594427 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.595981 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.596348 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.597334 (+ 986us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.600229 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.600235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.600238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.600280 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.607119 (+ 6839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.609290 (+ 2171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.609294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.609296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.609389 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.609740 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.609743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.609745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.609969 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.610106 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.610722 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.610855 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":75,"threads_started":1}
I20260807 15:15:16.612146 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.589572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.31.230.253:43185)
0807 15:15:16.590169 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.590174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.592306 (+ 2132us) server_negotiation.cc:407] Connection header received
0807 15:15:16.592355 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.592357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.592410 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.592500 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.593508 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.596722 (+ 3214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.598091 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.598467 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.598590 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.600701 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.600706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.600710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.600739 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.608730 (+ 7991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.610512 (+ 1782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.610516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.610517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.610611 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.610936 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.610939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.610941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.611148 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.611271 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.611925 (+ 654us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.612039 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":84,"threads_started":1}
I20260807 15:15:16.614070 8934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:16.617300 8934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:16.620108 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: NON_VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: true } } }
I20260807 15:15:16.620257 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:16.622212 10706 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: attempting to promote NON_VOTER 425b378830e04c9eb99cc5931824e4d5 to VOTER
I20260807 15:15:16.622571 10706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } }
I20260807 15:15:16.623788 9789 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Refusing update from remote peer 1ce63408ebb0481eb74bd908e2211fba: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:15:16.623792 10049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 LEARNER]: Refusing update from remote peer 1ce63408ebb0481eb74bd908e2211fba: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:15:16.624195 10707 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 15:15:16.624318 10706 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 15:15:16.626204 9979 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.626654 9789 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.626617 10049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.627854 9979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.627971 9979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:16.628122 9979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.628216 9979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:16.628333 9979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.628410 9979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:16.628396 10093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.628551 10093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:16.628755 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ce63408ebb0481eb74bd908e2211fba" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:16.628858 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:16.640654 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.635528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39951 (local address 127.0.0.1:39920)
0807 15:15:16.636237 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.636251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.636343 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.637108 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.637111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.637130 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:16.637227 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.637233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.638413 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.638417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.639203 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.639212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.640241 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.640244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.640327 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.640351 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.640415 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.640491 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":21,"thread_start_us":108,"threads_started":1}
I20260807 15:15:16.640674 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.636053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39920 (local address 127.31.230.193:39951)
0807 15:15:16.636637 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.636657 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.636681 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:15:16.636761 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.636766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.636978 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.637148 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:16.637323 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.638269 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.639340 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.640113 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.640251 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.640399 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.640453 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.640554 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":98,"threads_started":1}
I20260807 15:15:16.640889 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.634940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:32959 (local address 127.0.0.1:45508)
0807 15:15:16.635067 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.635084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.635166 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.636190 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.636196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.636218 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:16.636381 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.636390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.638211 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.638215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.639364 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.639374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.640490 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.640494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.640578 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.640601 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.640644 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.640712 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":3}
I20260807 15:15:16.641571 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.635159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.31.230.195:32959)
0807 15:15:16.635556 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.635573 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.635650 (+ 77us) server_negotiation.cc:407] Connection header received
0807 15:15:16.635760 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.635766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.636011 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.636735 (+ 724us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:16.636900 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.638076 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.639529 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.640353 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.641092 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.641220 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.641276 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.641365 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":138,"threads_started":1}
I20260807 15:15:16.643451 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.636037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.31.230.194:44365)
0807 15:15:16.636816 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.636828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.637116 (+ 288us) server_negotiation.cc:407] Connection header received
0807 15:15:16.637364 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.637370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.637572 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.637717 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:16.637941 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.639058 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.641486 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.642315 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.642598 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.642720 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.643215 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.643313 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":138,"threads_started":1}
I20260807 15:15:16.643692 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.635594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:44365 (local address 127.0.0.1:50752)
0807 15:15:16.636964 (+ 1370us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.636977 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.637192 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.637716 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.637719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.637738 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:16.637827 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.637832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.639555 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.639558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.641357 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.641367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.642787 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.642790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.642878 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.642900 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.642971 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.643052 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"mutex_wait_us":1,"thread_start_us":427,"threads_started":1}
I20260807 15:15:16.644130 10197 tablet_service.cc:1467] Processing CreateTablet for tablet 4adfd42022c74a5f96b6717bae6b7164 (DEFAULT_TABLE table=test-table [id=aee46f8f168e4162b18832fe9f06a8a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:16.644194 10588 tablet_service.cc:1467] Processing CreateTablet for tablet 4adfd42022c74a5f96b6717bae6b7164 (DEFAULT_TABLE table=test-table [id=aee46f8f168e4162b18832fe9f06a8a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:16.645110 10588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4adfd42022c74a5f96b6717bae6b7164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:16.645191 10197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4adfd42022c74a5f96b6717bae6b7164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:16.652364 10723 tablet_bootstrap.cc:492] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Bootstrap starting.
I20260807 15:15:16.653658 10724 tablet_bootstrap.cc:492] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Bootstrap starting.
I20260807 15:15:16.654277 10390 tablet_service.cc:1467] Processing CreateTablet for tablet 4adfd42022c74a5f96b6717bae6b7164 (DEFAULT_TABLE table=test-table [id=aee46f8f168e4162b18832fe9f06a8a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:16.654825 10723 tablet_bootstrap.cc:654] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:16.655475 10724 tablet_bootstrap.cc:654] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:16.655485 10390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4adfd42022c74a5f96b6717bae6b7164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:16.655680 10723 log.cc:824] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:16.656361 10724 log.cc:824] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:16.657878 10723 tablet_bootstrap.cc:492] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: No bootstrap required, opened a new log
I20260807 15:15:16.658085 10723 ts_tablet_manager.cc:1397] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:15:16.658612 10724 tablet_bootstrap.cc:492] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: No bootstrap required, opened a new log
I20260807 15:15:16.658823 10724 ts_tablet_manager.cc:1397] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:15:16.662132 10723 raft_consensus.cc:348] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.662436 10723 raft_consensus.cc:374] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:16.662509 10723 raft_consensus.cc:729] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded878401a1f46989497aec764dc1942, State: Initialized, Role: FOLLOWER
I20260807 15:15:16.662326 10724 raft_consensus.cc:348] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.662616 10724 raft_consensus.cc:374] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:16.662688 10724 raft_consensus.cc:729] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc22d5a2cf8472fa50f7a28c0a75706, State: Initialized, Role: FOLLOWER
I20260807 15:15:16.662984 10723 consensus_queue.cc:260] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.663147 10724 consensus_queue.cc:260] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.663883 10684 heartbeater.cc:503] Master 127.31.230.254:37793 was elected leader, sending a full tablet report...
I20260807 15:15:16.664168 10724 ts_tablet_manager.cc:1428] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260807 15:15:16.664696 10723 ts_tablet_manager.cc:1428] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:15:16.666438 10294 heartbeater.cc:503] Master 127.31.230.254:37793 was elected leader, sending a full tablet report...
I20260807 15:15:16.669216 10732 tablet_bootstrap.cc:492] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Bootstrap starting.
I20260807 15:15:16.671341 10732 tablet_bootstrap.cc:654] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:16.672086 10732 log.cc:824] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:16.674110 10732 tablet_bootstrap.cc:492] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: No bootstrap required, opened a new log
I20260807 15:15:16.674319 10732 ts_tablet_manager.cc:1397] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:15:16.677721 10732 raft_consensus.cc:348] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.677951 10732 raft_consensus.cc:374] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:16.678023 10732 raft_consensus.cc:729] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e40373a5dd4252878c99852d074b20, State: Initialized, Role: FOLLOWER
I20260807 15:15:16.678437 10732 consensus_queue.cc:260] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.679152 10485 heartbeater.cc:503] Master 127.31.230.254:37793 was elected leader, sending a full tablet report...
I20260807 15:15:16.679416 10732 ts_tablet_manager.cc:1428] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:15:16.749086 10734 raft_consensus.cc:482] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:16.749353 10734 raft_consensus.cc:504] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.750579 10734 leader_election.cc:288] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959), ded878401a1f46989497aec764dc1942 (127.31.230.193:39951)
I20260807 15:15:16.754395 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.751004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:35451 (local address 127.31.230.193:39951)
0807 15:15:16.751153 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.751160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.751178 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:15:16.751369 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.751373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.751422 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.751506 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:16.751806 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.752325 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.753361 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.753810 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.753941 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.754000 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.754183 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.754252 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:15:16.754417 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.750985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:53367 (local address 127.31.230.195:32959)
0807 15:15:16.751132 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.751137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.751266 (+ 129us) server_negotiation.cc:407] Connection header received
0807 15:15:16.751435 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.751438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.751488 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.751585 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:16.751884 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.752389 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.753383 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.753843 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.753976 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.754026 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.754212 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.754279 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:15:16.754420 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.750877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:32959 (local address 127.31.230.194:53367)
0807 15:15:16.751154 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.751170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.751308 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.751596 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.751599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.751615 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:16.751763 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.751768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.752507 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.752512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.753246 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.753254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.753970 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.753973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.754080 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.754104 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.754195 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.754267 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":68}
I20260807 15:15:16.754433 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.750877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39951 (local address 127.31.230.194:35451)
0807 15:15:16.751107 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.751122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:16.751228 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:16.751520 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:16.751523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:16.751539 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:16.751681 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.751686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.752443 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.752448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.753216 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:16.753224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.753934 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.753938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:16.754045 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.754073 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:15:16.754164 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:15:16.754236 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":2}
I20260807 15:15:16.755103 10617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adfd42022c74a5f96b6717bae6b7164" candidate_uuid: "11e40373a5dd4252878c99852d074b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" is_pre_election: true
I20260807 15:15:16.755107 10225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adfd42022c74a5f96b6717bae6b7164" candidate_uuid: "11e40373a5dd4252878c99852d074b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ded878401a1f46989497aec764dc1942" is_pre_election: true
I20260807 15:15:16.755440 10617 raft_consensus.cc:2454] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11e40373a5dd4252878c99852d074b20 in term 0.
I20260807 15:15:16.755441 10225 raft_consensus.cc:2454] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11e40373a5dd4252878c99852d074b20 in term 0.
I20260807 15:15:16.755939 10328 leader_election.cc:302] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11e40373a5dd4252878c99852d074b20, ded878401a1f46989497aec764dc1942; no voters:
I20260807 15:15:16.756251 10734 raft_consensus.cc:2781] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:16.756404 10734 raft_consensus.cc:482] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:16.756505 10734 raft_consensus.cc:3037] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:16.757664 10734 raft_consensus.cc:504] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.758037 10734 leader_election.cc:288] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [CANDIDATE]: Term 1 election: Requested vote from peers 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959), ded878401a1f46989497aec764dc1942 (127.31.230.193:39951)
I20260807 15:15:16.758335 10617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adfd42022c74a5f96b6717bae6b7164" candidate_uuid: "11e40373a5dd4252878c99852d074b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706"
I20260807 15:15:16.758389 10225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4adfd42022c74a5f96b6717bae6b7164" candidate_uuid: "11e40373a5dd4252878c99852d074b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ded878401a1f46989497aec764dc1942"
I20260807 15:15:16.758451 10617 raft_consensus.cc:3037] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:16.758525 10225 raft_consensus.cc:3037] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:16.759593 10617 raft_consensus.cc:2454] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11e40373a5dd4252878c99852d074b20 in term 1.
I20260807 15:15:16.759600 10225 raft_consensus.cc:2454] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11e40373a5dd4252878c99852d074b20 in term 1.
I20260807 15:15:16.759897 10326 leader_election.cc:302] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc22d5a2cf8472fa50f7a28c0a75706, 11e40373a5dd4252878c99852d074b20; no voters:
I20260807 15:15:16.760089 10734 raft_consensus.cc:2781] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:16.760347 10734 raft_consensus.cc:686] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 1 LEADER]: Becoming Leader. State: Replica: 11e40373a5dd4252878c99852d074b20, State: Running, Role: LEADER
I20260807 15:15:16.760769 10734 consensus_queue.cc:237] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } }
I20260807 15:15:16.764405 8933 catalog_manager.cc:5495] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11e40373a5dd4252878c99852d074b20 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "11e40373a5dd4252878c99852d074b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:16.795709 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.791718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.31.230.194:44365)
0807 15:15:16.791876 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.791881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.791901 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:15:16.792024 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.792027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.792100 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.792229 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:16.792509 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.793062 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.794021 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.794260 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.794375 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.794755 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:16.795297 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:16.795435 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.795474 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.795538 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:15:16.801863 10225 raft_consensus.cc:1261] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Refusing update from remote peer 11e40373a5dd4252878c99852d074b20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:16.801863 10617 raft_consensus.cc:1261] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Refusing update from remote peer 11e40373a5dd4252878c99852d074b20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:16.802680 10734 consensus_queue.cc:1035] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:16.802899 10735 consensus_queue.cc:1035] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:16.823069 8934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:16.823253 8934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:16.828339 8934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ce63408ebb0481eb74bd908e2211fba: Sending DeleteTablet for 3 replicas of tablet 4adfd42022c74a5f96b6717bae6b7164
I20260807 15:15:16.829195 10588 tablet_service.cc:1514] Processing DeleteTablet for tablet 4adfd42022c74a5f96b6717bae6b7164 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:16 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:45508
I20260807 15:15:16.829381 10197 tablet_service.cc:1514] Processing DeleteTablet for tablet 4adfd42022c74a5f96b6717bae6b7164 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:16 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:39920
I20260807 15:15:16.829875 10753 tablet_replica.cc:330] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: stopping tablet replica
I20260807 15:15:16.829836 10390 tablet_service.cc:1514] Processing DeleteTablet for tablet 4adfd42022c74a5f96b6717bae6b7164 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:16 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:50752
I20260807 15:15:16.830235 10753 raft_consensus.cc:2229] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:16.830562 10755 tablet_replica.cc:330] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: stopping tablet replica
I20260807 15:15:16.830565 10753 raft_consensus.cc:2258] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:16.830960 10752 tablet_replica.cc:330] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: stopping tablet replica
I20260807 15:15:16.831247 10752 raft_consensus.cc:2229] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:16.831547 10752 raft_consensus.cc:2258] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:16.831123 10755 raft_consensus.cc:2229] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:16.832096 10755 raft_consensus.cc:2258] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:16.832275 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 8895
I20260807 15:15:16.837440 10753 ts_tablet_manager.cc:1905] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:16.841163 10755 ts_tablet_manager.cc:1905] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:16.841163 10752 ts_tablet_manager.cc:1905] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:16.843940 10752 ts_tablet_manager.cc:1918] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:16.843940 10755 ts_tablet_manager.cc:1918] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:16.844061 10752 log.cc:1196] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/4adfd42022c74a5f96b6717bae6b7164
I20260807 15:15:16.844199 10755 log.cc:1196] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/4adfd42022c74a5f96b6717bae6b7164
I20260807 15:15:16.844439 10752 ts_tablet_manager.cc:1939] T 4adfd42022c74a5f96b6717bae6b7164 P 0bc22d5a2cf8472fa50f7a28c0a75706: Deleting consensus metadata
I20260807 15:15:16.844700 10755 ts_tablet_manager.cc:1939] T 4adfd42022c74a5f96b6717bae6b7164 P 11e40373a5dd4252878c99852d074b20: Deleting consensus metadata
I20260807 15:15:16.846319 10753 ts_tablet_manager.cc:1918] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:16.846462 10753 log.cc:1196] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/4adfd42022c74a5f96b6717bae6b7164
I20260807 15:15:16.846802 10753 ts_tablet_manager.cc:1939] T 4adfd42022c74a5f96b6717bae6b7164 P ded878401a1f46989497aec764dc1942: Deleting consensus metadata
W20260807 15:15:16.850185 10137 connection.cc:535] server connection from 127.0.0.1:39920 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:15:16.850490 10326 connection.cc:535] server connection from 127.0.0.1:50752 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:15:16.850586 10518 connection.cc:535] server connection from 127.0.0.1:45508 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:15:16.868603 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.831237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.31.230.253:43185)
0807 15:15:16.831477 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.831482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.849935 (+ 18453us) server_negotiation.cc:407] Connection header received
0807 15:15:16.850110 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.850114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.850180 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.850275 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.851239 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.851994 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.853191 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.853514 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.853663 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.855454 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.855458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.855460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.855486 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.857748 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.866577 (+ 8829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.866581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.866583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.866689 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.867005 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.867008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.867010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.867271 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.867399 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.868227 (+ 828us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.868376 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260807 15:15:16.870824 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.854897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58596 (local address 127.31.230.252:39035)
0807 15:15:16.855030 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.855035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.855836 (+ 801us) server_negotiation.cc:407] Connection header received
0807 15:15:16.856013 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.856016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.856068 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.856119 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.856946 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.857769 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.858995 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.859334 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.859410 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.866842 (+ 7432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.866846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.866849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.866878 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.868829 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.869309 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.869313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.869315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.869400 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.869874 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.869878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.869880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.870092 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.870193 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.870533 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.870655 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:15:16.871275 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.855154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.31.230.253:43185)
0807 15:15:16.855470 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.855474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.855486 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:15:16.855514 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.855518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.855562 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.855653 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.859043 (+ 3390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.860518 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.862674 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.863164 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.863297 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.867035 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.867041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.867047 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.867073 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.869315 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.869845 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.869850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.869852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.869938 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.870260 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.870266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.870269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.870485 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.870619 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.871015 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.871123 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260807 15:15:16.878782 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.831356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.31.230.252:39035)
0807 15:15:16.831601 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:16.831606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:16.868108 (+ 36502us) server_negotiation.cc:407] Connection header received
0807 15:15:16.870164 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:16.870167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:16.870234 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:16.870287 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:16.871316 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.872107 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.873273 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:16.873500 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:16.873710 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:16.875233 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:16.875235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:16.875237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:16.875259 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:16.876775 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.877242 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.877245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.877247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.877348 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:16.877768 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:16.877772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:16.877773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:16.877993 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:16.878094 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:16.878469 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 15:15:16.878619 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260807 15:15:17.815865 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:17.815311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:59599)
0807 15:15:17.815615 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:17.815705 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":111,"threads_started":1}
I20260807 15:15:17.816614 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:17.816010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:44981)
0807 15:15:17.816368 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:17.816467 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":145,"threads_started":1}
W20260807 15:15:17.816653 10485 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:37793 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
W20260807 15:15:17.817111 10684 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:37793 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:17.818882 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:17.818486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:39041)
0807 15:15:17.818690 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:17.818776 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
W20260807 15:15:17.819368 10294 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:37793 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:18.328290 10771 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ce63408ebb0481eb74bd908e2211fba)
I20260807 15:15:18.328483 10772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ce63408ebb0481eb74bd908e2211fba)
I20260807 15:15:18.328594 10771 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } }
I20260807 15:15:18.328711 10772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } }
I20260807 15:15:18.329813 10772 leader_election.cc:288] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793), ee1679942a1e47cbb568531139078826 (127.31.230.253:43185)
I20260807 15:15:18.329878 10771 leader_election.cc:288] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793), 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252:39035)
I20260807 15:15:18.330542 10776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.329995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50180)
0807 15:15:18.330321 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.330417 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":78,"threads_started":1}
W20260807 15:15:18.330724 10012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:18.330963 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.329996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50190)
0807 15:15:18.330761 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.330845 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":74,"threads_started":1}
W20260807 15:15:18.331246 9754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:18.331889 10776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.331554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50206)
0807 15:15:18.331759 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.331817 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260807 15:15:18.331993 10012 leader_election.cc:334] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793): Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:18.332465 10781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.332168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50214)
0807 15:15:18.332337 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.332404 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 15:15:18.332609 9754 leader_election.cc:334] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793): Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:18.335703 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.330776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:39035 (local address 127.0.0.1:58608)
0807 15:15:18.331151 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.331166 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.331278 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.331605 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.331609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.331634 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.331770 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.331779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.332865 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.332868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.334232 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.334244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.335205 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.335209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.335310 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.335340 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.335454 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.335537 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":63,"threads_started":1}
I20260807 15:15:18.335712 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.330574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.31.230.252:39035)
0807 15:15:18.331046 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.331051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.331270 (+ 219us) server_negotiation.cc:407] Connection header received
0807 15:15:18.331378 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.331385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.331464 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.331593 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.331919 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.332753 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.334401 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.335087 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.335260 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.335322 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.335472 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.335551 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":76,"threads_started":1}
I20260807 15:15:18.336176 10049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1679942a1e47cbb568531139078826" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "425b378830e04c9eb99cc5931824e4d5" is_pre_election: true
I20260807 15:15:18.336352 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.330118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:43185 (local address 127.0.0.1:51744)
0807 15:15:18.330419 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.330439 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.330571 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.331743 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.331746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.331768 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.331889 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.331901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.333493 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.333497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.334877 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.334889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.335844 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.335848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.335966 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.335994 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.336053 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.336131 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":49,"threads_started":1}
I20260807 15:15:18.336452 10049 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1679942a1e47cbb568531139078826 in term 1.
I20260807 15:15:18.336658 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.330242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.31.230.253:43185)
0807 15:15:18.330509 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.330514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.330545 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:15:18.330765 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.330771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.330897 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.332348 (+ 1451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.332463 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.333353 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.335015 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.335675 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.336295 (+ 620us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.336350 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.336395 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.336478 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260807 15:15:18.336861 9753 leader_election.cc:302] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 425b378830e04c9eb99cc5931824e4d5, ee1679942a1e47cbb568531139078826; no voters: 1ce63408ebb0481eb74bd908e2211fba
I20260807 15:15:18.337117 10771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:15:18.337080 9789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "425b378830e04c9eb99cc5931824e4d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ee1679942a1e47cbb568531139078826" is_pre_election: true
I20260807 15:15:18.337182 10771 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ce63408ebb0481eb74bd908e2211fba)
I20260807 15:15:18.337229 10771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:15:18.338434 10771 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } }
I20260807 15:15:18.338711 9789 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 425b378830e04c9eb99cc5931824e4d5 in current term 2: Already voted for candidate ee1679942a1e47cbb568531139078826 in this term.
I20260807 15:15:18.338935 10771 leader_election.cc:288] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [CANDIDATE]: Term 2 election: Requested vote from peers 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793), 425b378830e04c9eb99cc5931824e4d5 (127.31.230.252:39035)
I20260807 15:15:18.339323 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.339060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50226)
0807 15:15:18.339190 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.339257 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:15:18.339363 10049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1679942a1e47cbb568531139078826" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "425b378830e04c9eb99cc5931824e4d5"
I20260807 15:15:18.339537 10049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:15:18.339473 10013 leader_election.cc:302] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 425b378830e04c9eb99cc5931824e4d5; no voters: 1ce63408ebb0481eb74bd908e2211fba, ee1679942a1e47cbb568531139078826
I20260807 15:15:18.340107 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.339856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50232)
0807 15:15:18.339977 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.340028 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 15:15:18.340226 9754 leader_election.cc:334] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793): Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
I20260807 15:15:18.340824 10049 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1679942a1e47cbb568531139078826 in term 2.
I20260807 15:15:18.340941 10772 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:15:18.341251 9753 leader_election.cc:302] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 425b378830e04c9eb99cc5931824e4d5, ee1679942a1e47cbb568531139078826; no voters: 1ce63408ebb0481eb74bd908e2211fba
I20260807 15:15:18.341585 10771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:15:18.341842 10771 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [term 2 LEADER]: Becoming Leader. State: Replica: ee1679942a1e47cbb568531139078826, State: Running, Role: LEADER
I20260807 15:15:18.342115 10771 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } }
I20260807 15:15:18.343252 10786 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1679942a1e47cbb568531139078826. Latest consensus state: current_term: 2 leader_uuid: "ee1679942a1e47cbb568531139078826" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:18.343386 10786 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:18.343726 10789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:18.346051 10789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:18.346481 10789 catalog_manager.cc:1259] Loaded cluster ID: 2ff8203034124349b90f11131fc8c229
I20260807 15:15:18.346554 10789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:18.347115 10789 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:18.347517 10789 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Loaded TSK: 0
I20260807 15:15:18.348207 10789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:18.398581 9778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:18.401783 9778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:18.404968 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.404593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50240)
0807 15:15:18.404813 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.404884 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 15:15:18.404994 10049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [term 2 FOLLOWER]: Refusing update from remote peer ee1679942a1e47cbb568531139078826: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 15:15:18.405608 10786 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 15:15:18.405757 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.405537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.0.0.1:50242)
0807 15:15:18.405654 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.405702 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 15:15:18.405901 9754 consensus_peers.cc:487] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 -> Peer 1ce63408ebb0481eb74bd908e2211fba (127.31.230.254:37793): Couldn't send request to peer 1ce63408ebb0481eb74bd908e2211fba. Status: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:15:18.407187 10772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1679942a1e47cbb568531139078826. Latest consensus state: current_term: 2 leader_uuid: "ee1679942a1e47cbb568531139078826" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:18.407313 10772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:18.408500 10786 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ee1679942a1e47cbb568531139078826" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:18.408629 10786 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:18.408823 10772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ee1679942a1e47cbb568531139078826" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1ce63408ebb0481eb74bd908e2211fba" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37793 } } peers { permanent_uuid: "ee1679942a1e47cbb568531139078826" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 43185 } attrs { promote: false } } peers { permanent_uuid: "425b378830e04c9eb99cc5931824e4d5" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 39035 } attrs { promote: false } } }
I20260807 15:15:18.408950 10772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b378830e04c9eb99cc5931824e4d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:15:18.419157 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.414675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:32959 (local address 127.0.0.1:45524)
0807 15:15:18.414966 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.414979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.415093 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.415454 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.415458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.415480 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.415595 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.415603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.416697 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.416700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.417864 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.417875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.418777 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.418781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.418873 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.418902 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.418952 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.419018 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1}
I20260807 15:15:18.419680 10781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.414845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39951 (local address 127.0.0.1:39934)
0807 15:15:18.415090 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.415104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.415191 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.416196 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.416199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.416216 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.416299 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.416304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.417348 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.417352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.418394 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.418405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.419279 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.419285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.419376 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.419433 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.419493 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.419558 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":31}
I20260807 15:15:18.419754 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.414744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45524 (local address 127.31.230.195:32959)
0807 15:15:18.415078 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.415083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.415101 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:15:18.415265 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.415270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.415325 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.415716 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.415837 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.416608 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.418010 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.418636 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.419481 (+ 845us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.419532 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.419564 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.419626 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":92,"threads_started":1}
I20260807 15:15:18.419742 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.415357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.31.230.193:39951)
0807 15:15:18.415884 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.415889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.415969 (+ 80us) server_negotiation.cc:407] Connection header received
0807 15:15:18.416025 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.416028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.416076 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.416396 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.416498 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.417237 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.418542 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.419143 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.419439 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.419496 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.419534 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.419611 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":79,"threads_started":1}
I20260807 15:15:18.420176 10798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.415158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:44365 (local address 127.0.0.1:50772)
0807 15:15:18.415636 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.415654 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.415746 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.416331 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.416334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.416351 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.416448 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.416454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.417492 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.417495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.418562 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.418571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.419629 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.419632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.419721 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.419891 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.419967 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.420040 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":124,"threads_started":1}
I20260807 15:15:18.420325 10588 tablet_service.cc:1467] Processing CreateTablet for tablet 4df868bac6324f40b01a87e89abf839f (DEFAULT_TABLE table=test-table [id=b344b0c750c04eafb78f216af6834398]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:18.420599 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.415561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.31.230.194:44365)
0807 15:15:18.416084 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.416088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.416103 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:18.416159 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.416162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.416210 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.416320 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.416566 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.417365 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.418713 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.419358 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.419510 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.419563 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.420333 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.420428 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":83,"threads_started":1}
I20260807 15:15:18.420568 10197 tablet_service.cc:1467] Processing CreateTablet for tablet 4df868bac6324f40b01a87e89abf839f (DEFAULT_TABLE table=test-table [id=b344b0c750c04eafb78f216af6834398]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:18.420662 10588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df868bac6324f40b01a87e89abf839f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:18.420816 10197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df868bac6324f40b01a87e89abf839f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:18.423400 10801 tablet_bootstrap.cc:492] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: Bootstrap starting.
I20260807 15:15:18.423723 10390 tablet_service.cc:1467] Processing CreateTablet for tablet 4df868bac6324f40b01a87e89abf839f (DEFAULT_TABLE table=test-table [id=b344b0c750c04eafb78f216af6834398]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:18.423955 10390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df868bac6324f40b01a87e89abf839f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:18.424383 10801 tablet_bootstrap.cc:654] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:18.425043 10802 tablet_bootstrap.cc:492] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Bootstrap starting.
I20260807 15:15:18.425961 10801 tablet_bootstrap.cc:492] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: No bootstrap required, opened a new log
I20260807 15:15:18.426034 10802 tablet_bootstrap.cc:654] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:18.426167 10801 ts_tablet_manager.cc:1397] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:18.426755 10801 raft_consensus.cc:348] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.426811 10804 tablet_bootstrap.cc:492] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Bootstrap starting.
I20260807 15:15:18.426882 10801 raft_consensus.cc:374] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:18.426929 10801 raft_consensus.cc:729] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded878401a1f46989497aec764dc1942, State: Initialized, Role: FOLLOWER
I20260807 15:15:18.427062 10801 consensus_queue.cc:260] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.427464 10801 ts_tablet_manager.cc:1428] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:18.427712 10804 tablet_bootstrap.cc:654] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:18.428993 10804 tablet_bootstrap.cc:492] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: No bootstrap required, opened a new log
I20260807 15:15:18.429106 10804 ts_tablet_manager.cc:1397] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:18.430095 10802 tablet_bootstrap.cc:492] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: No bootstrap required, opened a new log
I20260807 15:15:18.430200 10802 ts_tablet_manager.cc:1397] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 15:15:18.430253 10295 heartbeater.cc:503] Master 127.31.230.253:43185 was elected leader, sending a full tablet report...
I20260807 15:15:18.430322 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.427755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:37385)
0807 15:15:18.430127 (+ 2372us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.430211 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2277,"thread_start_us":98,"threads_started":1}
I20260807 15:15:18.430640 10802 raft_consensus.cc:348] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.430763 10802 raft_consensus.cc:374] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:18.430800 10802 raft_consensus.cc:729] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc22d5a2cf8472fa50f7a28c0a75706, State: Initialized, Role: FOLLOWER
I20260807 15:15:18.430931 10802 consensus_queue.cc:260] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.431195 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.430748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:50981)
0807 15:15:18.431043 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.431098 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260807 15:15:18.431332 10802 ts_tablet_manager.cc:1428] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:18.432243 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.431805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:53161)
0807 15:15:18.432089 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.432159 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
I20260807 15:15:18.429571 10804 raft_consensus.cc:348] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.432787 10804 raft_consensus.cc:374] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:18.432834 10804 raft_consensus.cc:729] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e40373a5dd4252878c99852d074b20, State: Initialized, Role: FOLLOWER
I20260807 15:15:18.432955 10685 heartbeater.cc:503] Master 127.31.230.253:43185 was elected leader, sending a full tablet report...
I20260807 15:15:18.432972 10804 consensus_queue.cc:260] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.433168 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.432692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:35801)
0807 15:15:18.433039 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.433095 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 15:15:18.433372 10804 ts_tablet_manager.cc:1428] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:15:18.434329 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.433867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:36495)
0807 15:15:18.434162 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.434221 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":94,"threads_started":1}
I20260807 15:15:18.434959 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.434769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:41821)
0807 15:15:18.434864 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.434908 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:15:18.435514 10486 heartbeater.cc:503] Master 127.31.230.253:43185 was elected leader, sending a full tablet report...
I20260807 15:15:18.550858 10807 raft_consensus.cc:482] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:18.551126 10807 raft_consensus.cc:504] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.552313 10807 leader_election.cc:288] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959), 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365)
I20260807 15:15:18.556329 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.552741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:36027 (local address 127.31.230.194:44365)
0807 15:15:18.552871 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.552877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.552892 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:18.553080 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.553082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.553124 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.553211 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.553494 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.554029 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.555279 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.555676 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.555813 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.555865 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.556117 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.556195 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:15:18.556330 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.552620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:44365 (local address 127.31.230.193:36027)
0807 15:15:18.552797 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.552813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.552921 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.553223 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.553225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.553243 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.553370 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.553376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.554160 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.554164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.555123 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.555133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.555820 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.555824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.555955 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.556001 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.556097 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.556178 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":78,"spinlock_wait_cycles":1792}
I20260807 15:15:18.556336 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.552658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:43007 (local address 127.31.230.195:32959)
0807 15:15:18.552776 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.552782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.552836 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:15:18.553019 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.553022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.553072 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.553164 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:18.553488 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.554048 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.555276 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.555665 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.555803 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.555867 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.556130 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.556198 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:15:18.556345 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.552528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:32959 (local address 127.31.230.193:43007)
0807 15:15:18.552733 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.552761 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:18.552901 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:18.553176 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:18.553179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:18.553195 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:18.553366 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.553373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.554174 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.554178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.555137 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:18.555146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.555828 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.555832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:18.555985 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.556019 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 15:15:18.556110 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:15:18.556193 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":93}
I20260807 15:15:18.556874 10617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df868bac6324f40b01a87e89abf839f" candidate_uuid: "ded878401a1f46989497aec764dc1942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" is_pre_election: true
I20260807 15:15:18.557081 10617 raft_consensus.cc:2454] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ded878401a1f46989497aec764dc1942 in term 0.
I20260807 15:15:18.557057 10418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df868bac6324f40b01a87e89abf839f" candidate_uuid: "ded878401a1f46989497aec764dc1942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11e40373a5dd4252878c99852d074b20" is_pre_election: true
I20260807 15:15:18.557320 10418 raft_consensus.cc:2454] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ded878401a1f46989497aec764dc1942 in term 0.
I20260807 15:15:18.557576 10133 leader_election.cc:302] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc22d5a2cf8472fa50f7a28c0a75706, ded878401a1f46989497aec764dc1942; no voters:
I20260807 15:15:18.557910 10807 raft_consensus.cc:2781] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:18.557976 10807 raft_consensus.cc:482] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:18.558004 10807 raft_consensus.cc:3037] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:18.559089 10807 raft_consensus.cc:504] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.559475 10807 leader_election.cc:288] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [CANDIDATE]: Term 1 election: Requested vote from peers 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959), 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365)
I20260807 15:15:18.559841 10418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df868bac6324f40b01a87e89abf839f" candidate_uuid: "ded878401a1f46989497aec764dc1942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11e40373a5dd4252878c99852d074b20"
I20260807 15:15:18.559894 10617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df868bac6324f40b01a87e89abf839f" candidate_uuid: "ded878401a1f46989497aec764dc1942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706"
I20260807 15:15:18.559973 10418 raft_consensus.cc:3037] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:18.560021 10617 raft_consensus.cc:3037] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:18.560963 10617 raft_consensus.cc:2454] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ded878401a1f46989497aec764dc1942 in term 1.
I20260807 15:15:18.561051 10418 raft_consensus.cc:2454] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ded878401a1f46989497aec764dc1942 in term 1.
I20260807 15:15:18.561277 10133 leader_election.cc:302] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc22d5a2cf8472fa50f7a28c0a75706, ded878401a1f46989497aec764dc1942; no voters:
I20260807 15:15:18.561461 10807 raft_consensus.cc:2781] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:18.561748 10807 raft_consensus.cc:686] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 1 LEADER]: Becoming Leader. State: Replica: ded878401a1f46989497aec764dc1942, State: Running, Role: LEADER
I20260807 15:15:18.562172 10807 consensus_queue.cc:237] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } }
I20260807 15:15:18.562455 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.562187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:35067)
0807 15:15:18.562317 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.562376 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 15:15:18.564597 9778 catalog_manager.cc:5495] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 reported cstate change: term changed from 0 to 1, leader changed from <none> to ded878401a1f46989497aec764dc1942 (127.31.230.193). New cstate: current_term: 1 leader_uuid: "ded878401a1f46989497aec764dc1942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ded878401a1f46989497aec764dc1942" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 } health_report { overall_health: UNKNOWN } } }
W20260807 15:15:18.586199 10686 tablet.cc:2369] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:18.596496 10488 tablet.cc:2369] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:18.613844 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.609467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.31.230.193:39951)
0807 15:15:18.609631 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:18.609636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:18.609651 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:18.609806 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:18.609809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:18.609878 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:18.610006 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:18.610281 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.610847 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.612129 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:18.612390 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:18.612500 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:18.612815 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:18.613360 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:18.613463 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:18.613614 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:15:18.613676 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 15:15:18.614228 10297 tablet.cc:2369] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:18.617376 10617 raft_consensus.cc:1261] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Refusing update from remote peer ded878401a1f46989497aec764dc1942: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:18.617424 10418 raft_consensus.cc:1261] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Refusing update from remote peer ded878401a1f46989497aec764dc1942: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:18.618067 10815 consensus_queue.cc:1035] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc22d5a2cf8472fa50f7a28c0a75706" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:18.618239 10807 consensus_queue.cc:1035] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11e40373a5dd4252878c99852d074b20" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:18.620815 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.620056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:38311)
0807 15:15:18.620628 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.620726 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482}
I20260807 15:15:18.621515 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.621208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:57205)
0807 15:15:18.621354 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.621428 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 15:15:18.626655 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.626341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.195:49775)
0807 15:15:18.626506 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.626564 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 15:15:18.626981 10824 mvcc.cc:205] Tried to move back new op lower bound from 7315929983449665536 to 7315929983233245184. Current Snapshot: MvccSnapshot[applied={T|T < 7315929983449665536}]
I20260807 15:15:18.627907 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.625972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.194:58543)
0807 15:15:18.627762 (+ 1790us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.627825 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1706}
I20260807 15:15:18.628216 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.627914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:46833)
0807 15:15:18.628080 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.628136 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 15:15:18.631060 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.630772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37793 (local address 127.31.230.193:41899)
0807 15:15:18.630931 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.630988 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 15:15:18.636529 9779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:18.636724 9779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:18.641686 9779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee1679942a1e47cbb568531139078826: Sending DeleteTablet for 3 replicas of tablet 4df868bac6324f40b01a87e89abf839f
I20260807 15:15:18.642443 10588 tablet_service.cc:1514] Processing DeleteTablet for tablet 4df868bac6324f40b01a87e89abf839f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:18 UTC) from {username='oryx', principal='oryx/127.31.230.253@KRBTEST.COM'} at 127.0.0.1:45524
I20260807 15:15:18.642467 10197 tablet_service.cc:1514] Processing DeleteTablet for tablet 4df868bac6324f40b01a87e89abf839f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:18 UTC) from {username='oryx', principal='oryx/127.31.230.253@KRBTEST.COM'} at 127.0.0.1:39934
I20260807 15:15:18.642634 10390 tablet_service.cc:1514] Processing DeleteTablet for tablet 4df868bac6324f40b01a87e89abf839f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:18 UTC) from {username='oryx', principal='oryx/127.31.230.253@KRBTEST.COM'} at 127.0.0.1:50772
I20260807 15:15:18.642894 10836 tablet_replica.cc:330] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942: stopping tablet replica
I20260807 15:15:18.643035 10836 raft_consensus.cc:2229] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:18.643062 10841 tablet_replica.cc:330] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: stopping tablet replica
I20260807 15:15:18.643323 10841 raft_consensus.cc:2229] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:18.643527 10841 raft_consensus.cc:2258] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:18.643525 10836 raft_consensus.cc:2258] T 4df868bac6324f40b01a87e89abf839f P ded878401a1f46989497aec764dc1942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:18.643759 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 9975
I20260807 15:15:18.646193 10837 tablet_replica.cc:330] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: stopping tablet replica
I20260807 15:15:18.646342 10837 raft_consensus.cc:2229] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:18.646484 10837 raft_consensus.cc:2258] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:18.648320 10837 ts_tablet_manager.cc:1905] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:18.650734 10841 ts_tablet_manager.cc:1905] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:18.651588 10837 ts_tablet_manager.cc:1918] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:18.651674 10837 log.cc:1196] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/4df868bac6324f40b01a87e89abf839f
I20260807 15:15:18.652010 10837 ts_tablet_manager.cc:1939] T 4df868bac6324f40b01a87e89abf839f P 0bc22d5a2cf8472fa50f7a28c0a75706: Deleting consensus metadata
I20260807 15:15:18.652920 9754 catalog_manager.cc:4841] TS 0bc22d5a2cf8472fa50f7a28c0a75706 (127.31.230.195:32959): tablet 4df868bac6324f40b01a87e89abf839f (table test-table [id=b344b0c750c04eafb78f216af6834398]) successfully deleted
I20260807 15:15:18.653824 10841 ts_tablet_manager.cc:1918] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:18.653929 10841 log.cc:1196] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/4df868bac6324f40b01a87e89abf839f
I20260807 15:15:18.654181 10841 ts_tablet_manager.cc:1939] T 4df868bac6324f40b01a87e89abf839f P 11e40373a5dd4252878c99852d074b20: Deleting consensus metadata
I20260807 15:15:18.654870 9753 catalog_manager.cc:4841] TS 11e40373a5dd4252878c99852d074b20 (127.31.230.194:44365): tablet 4df868bac6324f40b01a87e89abf839f (table test-table [id=b344b0c750c04eafb78f216af6834398]) successfully deleted
I20260807 15:15:18.661015 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 10304
I20260807 15:15:18.661294 10798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.660958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39951 (local address 127.0.0.1:39950)
0807 15:15:18.661105 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.661158 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:39951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 15:15:18.661513 9756 catalog_manager.cc:4584] TS ded878401a1f46989497aec764dc1942 (127.31.230.193:39951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4df868bac6324f40b01a87e89abf839f: Network error: Client connection negotiation failed: client connection to 127.31.230.193:39951: connect: Connection refused (error 111)
I20260807 15:15:18.673288 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 10492
I20260807 15:15:18.688946 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 9584
I20260807 15:15:18.701802 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 9727
2026-08-07T15:15:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5946 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:18 dist-test-slave-gkw7 krb5kdc[10845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:18 dist-test-slave-gkw7 krb5kdc[10845](info): set up 2 sockets
Aug 07 15:15:18 dist-test-slave-gkw7 krb5kdc[10845](info): commencing operation
krb5kdc: starting...
W20260807 15:15:19.964675 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab.
Entry for principal oryx/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab.
I20260807 15:15:20.100613 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:36387
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab
--principal=oryx/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:36387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:20.214150 10860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:20.214448 10860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:20.214522 10860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:20.218497 10860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:20.218588 10860 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:15:20.218621 10860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:20.218663 10860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:20.218698 10860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:20.218762 10860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:20.223960 10860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:36387
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.254.keytab
--principal=oryx/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:20.225085 10860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:20.226192 10860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:20.235087 10874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:20.235090 10868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:20.235188 10860 server_base.cc:1034] running on GCE node
W20260807 15:15:20.235090 10867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:20.235980 10860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:20.237179 10860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:20.238375 10860 hybrid_clock.cc:648] HybridClock initialized: now 1786115720238344 us; error 44 us; skew 500 ppm
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:20.241075 10860 init.cc:377] Logged in from keytab as oryx/127.31.230.254@KRBTEST.COM (short username oryx)
I20260807 15:15:20.242100 10860 webserver.cc:460] Webserver started at http://127.31.230.254:33627/ using document root <none> and password file <none>
I20260807 15:15:20.242671 10860 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:20.242785 10860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:20.243032 10860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:20.245083 10860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "8270fdb0ff383e9beb891353efddfbc8"
server_key_iv: "90bd406b2fd4f89c153714b2f1fb1c9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.245731 10860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "8270fdb0ff383e9beb891353efddfbc8"
server_key_iv: "90bd406b2fd4f89c153714b2f1fb1c9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.249446 10860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:20.251957 10882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:20.253178 10860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:20.253335 10860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "8270fdb0ff383e9beb891353efddfbc8"
server_key_iv: "90bd406b2fd4f89c153714b2f1fb1c9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.253458 10860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:20.265319 10860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:20.268321 10860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:20.268543 10860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:20.276865 10860 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:36387
I20260807 15:15:20.276889 10968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:36387 every 8 connection(s)
I20260807 15:15:20.277952 10860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:20.280793 10969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:20.286616 10969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Bootstrap starting.
I20260807 15:15:20.287617 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 10860
I20260807 15:15:20.287763 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:20.288093 32667 external_mini_cluster.cc:1428] Setting key a85ad79ad51214b1c1a33979c5f7d1e2
I20260807 15:15:20.289740 10969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:20.290498 10969 log.cc:824] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:20.292380 10969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: No bootstrap required, opened a new log
I20260807 15:15:20.296044 10969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 36387 } }
I20260807 15:15:20.296301 10969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:20.296366 10969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f1db0a1c2444dc9a8f48e9095d9b7a, State: Initialized, Role: FOLLOWER
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:20.296813 10969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 36387 } }
I20260807 15:15:20.296970 10969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:20.297037 10969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:20.297168 10969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:20.298237 10969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 36387 } }
I20260807 15:15:20.298648 10969 leader_election.cc:302] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56f1db0a1c2444dc9a8f48e9095d9b7a; no voters:
I20260807 15:15:20.299023 10969 leader_election.cc:288] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:20.299146 10976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:20.299384 10976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [term 1 LEADER]: Becoming Leader. State: Replica: 56f1db0a1c2444dc9a8f48e9095d9b7a, State: Running, Role: LEADER
I20260807 15:15:20.299707 10976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 36387 } }
I20260807 15:15:20.300441 10969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:20.301738 10979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56f1db0a1c2444dc9a8f48e9095d9b7a. Latest consensus state: current_term: 1 leader_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 36387 } } }
I20260807 15:15:20.301738 10977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f1db0a1c2444dc9a8f48e9095d9b7a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 36387 } } }
I20260807 15:15:20.301918 10977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:20.301918 10979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:20.302538 10988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:20.303766 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.289264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.31.230.254:36387)
0807 15:15:20.289768 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:20.289805 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:20.289853 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:15:20.290536 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:20.290554 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:20.290939 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:20.291291 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:20.292114 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.293168 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.294100 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.294593 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.294775 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.297624 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:20.297629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:20.297642 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:20.297669 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:20.299721 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.300245 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.300251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.300257 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.300402 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.300729 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.300733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.300735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.301336 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:20.302360 (+ 1024us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:20.302544 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 15:15:20.302897 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":358,"thread_start_us":172,"threads_started":1}
I20260807 15:15:20.305737 10988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:20.310952 10988 catalog_manager.cc:1347] Generated new cluster ID: 51681420bd2d43dba982975c89181f9b
I20260807 15:15:20.311017 10988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:20.334365 10988 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:20.335364 10988 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:20.347393 10988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Generated new TSK 0
I20260807 15:15:20.348131 10988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab.
Entry for principal oryx/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab.
I20260807 15:15:20.385346 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:36387
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:20.501506 11002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:20.501884 11002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:20.501969 11002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:20.506062 11002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:20.506119 11002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:20.506289 11002 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:15:20.506419 11002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:20.512130 11002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:20.513230 11002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:20.514344 11002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:20.524606 11010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:20.524622 11008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:20.524622 11013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:20.524681 11002 server_base.cc:1034] running on GCE node
I20260807 15:15:20.525202 11002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:20.525959 11002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:20.527134 11002 hybrid_clock.cc:648] HybridClock initialized: now 1786115720527116 us; error 31 us; skew 500 ppm
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:20.530076 11002 init.cc:377] Logged in from keytab as oryx/127.31.230.193@KRBTEST.COM (short username oryx)
I20260807 15:15:20.530993 11002 webserver.cc:460] Webserver started at http://127.31.230.193:44135/ using document root <none> and password file <none>
I20260807 15:15:20.531647 11002 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:20.531746 11002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:20.531997 11002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:20.534263 11002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "93c8544156d34bcf88e1e3345f34658f"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "ff63bbb19f97766adc753242fbcb72ef"
server_key_iv: "2575991f72f8c23af6ec0d0409150db2"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.534859 11002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "93c8544156d34bcf88e1e3345f34658f"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "ff63bbb19f97766adc753242fbcb72ef"
server_key_iv: "2575991f72f8c23af6ec0d0409150db2"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.538683 11002 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:15:20.541177 11023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:20.542444 11002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:20.542583 11002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "93c8544156d34bcf88e1e3345f34658f"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "ff63bbb19f97766adc753242fbcb72ef"
server_key_iv: "2575991f72f8c23af6ec0d0409150db2"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.542694 11002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:20.558727 11002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:20.561652 11002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:20.561875 11002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:20.563315 11002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:20.563393 11002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:20.563469 11002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:20.563514 11002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:20.574330 11002 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:36233
I20260807 15:15:20.574402 11176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:36233 every 8 connection(s)
I20260807 15:15:20.575440 11002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:20.577991 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.564210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:39667 (local address 127.31.230.254:36387)
0807 15:15:20.564369 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:20.564374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:20.565295 (+ 921us) server_negotiation.cc:407] Connection header received
0807 15:15:20.566475 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:20.566479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:20.566538 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:20.566649 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:20.568235 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.569144 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.570534 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.570791 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.570913 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.574100 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:20.574104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:20.574106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:20.574139 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:20.575761 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.576387 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.576391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.576393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.576479 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.576846 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.576850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.576852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.577043 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:20.577154 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:20.577687 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 15:15:20.577818 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:15:20.578732 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.564662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.31.230.193:39667)
0807 15:15:20.565123 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.565174 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:20.566218 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:20.566843 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:20.566853 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:20.567336 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:20.568011 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.568038 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.569314 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.569320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.570400 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.570412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.570968 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.570972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.571152 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.572306 (+ 1154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:20.572322 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:20.573920 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:20.575881 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.575887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.575902 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.576262 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.576576 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.576580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.576582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.576733 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.577154 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:20.577161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:20.577399 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 15:15:20.577623 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 15:15:20.577942 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":135,"threads_started":1}
I20260807 15:15:20.580051 11177 heartbeater.cc:347] Connected to a master server at 127.31.230.254:36387
I20260807 15:15:20.580215 11177 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:20.580739 11177 heartbeater.cc:511] Master 127.31.230.254:36387 requested a full tablet report, sending...
I20260807 15:15:20.582386 10907 ts_manager.cc:194] Registered new tserver with Master: 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233)
I20260807 15:15:20.583217 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 11002
I20260807 15:15:20.583324 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:20.583655 32667 external_mini_cluster.cc:1428] Setting key d549919bb5bd5c40f65f1868d1e158c5
I20260807 15:15:20.583797 10907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:39667
I20260807 15:15:20.590284 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.582315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.31.230.193:51693)
0807 15:15:20.582592 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.582607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:20.582711 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:20.583005 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:20.583008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:20.583223 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:20.583341 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.583349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.584305 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.584308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.585239 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.585249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.585820 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.585824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.585917 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.586479 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:20.586491 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:20.586802 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:20.588580 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.588585 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.588589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.588857 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.589133 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.589136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.589138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.589261 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.589840 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:20.589842 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:20.589920 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 15:15:20.590007 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:15:20.590116 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":5,"thread_start_us":104,"threads_started":1}
I20260807 15:15:20.590745 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.582388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:51693 (local address 127.31.230.254:36387)
0807 15:15:20.582523 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:20.582528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:20.582706 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:15:20.582805 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:20.582807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:20.582849 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:20.582965 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:20.583477 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.584159 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.585370 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.585689 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.585802 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.586949 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:20.586955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:20.586961 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:20.586993 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:20.588463 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.588961 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.588964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.588965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.589040 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.589467 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.589472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.589476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.589740 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:20.589847 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:20.590091 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 15:15:20.590232 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for oryx/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab.
Entry for principal oryx/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab.
I20260807 15:15:20.616696 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:36387
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:20.730603 11191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:20.730862 11191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:20.730906 11191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:20.736408 11191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:20.736472 11191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:20.736511 11191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:15:20.736544 11191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:20.741677 11191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:20.742693 11191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:20.743737 11191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:20.753643 11198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:20.753657 11204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:20.753643 11199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:20.753876 11191 server_base.cc:1034] running on GCE node
I20260807 15:15:20.754357 11191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:20.755024 11191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:20.756199 11191 hybrid_clock.cc:648] HybridClock initialized: now 1786115720756164 us; error 48 us; skew 500 ppm
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:20.759246 11191 init.cc:377] Logged in from keytab as oryx/127.31.230.194@KRBTEST.COM (short username oryx)
I20260807 15:15:20.760167 11191 webserver.cc:460] Webserver started at http://127.31.230.194:42541/ using document root <none> and password file <none>
I20260807 15:15:20.760767 11191 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:20.760864 11191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:20.761114 11191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:20.763437 11191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "512b8289b81f499ea3023798555c1af0"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "e415332b0a42bebc913f47dc4e686203"
server_key_iv: "a0ce933b4d5195d115de5aae9a45ee73"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.764020 11191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "512b8289b81f499ea3023798555c1af0"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "e415332b0a42bebc913f47dc4e686203"
server_key_iv: "a0ce933b4d5195d115de5aae9a45ee73"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.767861 11191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 15:15:20.770378 11218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:20.771662 11191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:20.771813 11191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "512b8289b81f499ea3023798555c1af0"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "e415332b0a42bebc913f47dc4e686203"
server_key_iv: "a0ce933b4d5195d115de5aae9a45ee73"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.771927 11191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:20.784462 11191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:20.787509 11191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:20.787725 11191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:20.789175 11191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:20.789258 11191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:20.789332 11191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:20.789386 11191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:20.800563 11191 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:33337
I20260807 15:15:20.800596 11386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:33337 every 8 connection(s)
I20260807 15:15:20.802078 11191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:20.803802 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 11191
I20260807 15:15:20.803941 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:20.804224 32667 external_mini_cluster.cc:1428] Setting key ce3f190120689496bb156df664424829
I20260807 15:15:20.804751 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.790151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:44691 (local address 127.31.230.254:36387)
0807 15:15:20.790289 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:20.790293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:20.791386 (+ 1093us) server_negotiation.cc:407] Connection header received
0807 15:15:20.792461 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:20.792465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:20.792530 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:20.792594 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:20.794528 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.795436 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.797267 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.797638 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.797792 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.800634 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:20.800640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:20.800644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:20.800679 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:20.802359 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.803181 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.803186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.803188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.803284 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.803637 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.803642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.803644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.803840 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:20.803947 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:20.804442 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 15:15:20.804575 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:15:20.805722 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.790672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.31.230.194:44691)
0807 15:15:20.791200 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.791243 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:20.792210 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:20.793089 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:20.793097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:20.793625 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:20.794343 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.794367 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.795581 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.795584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.797119 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.797131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.797805 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.797808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.797947 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.798567 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:20.798594 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:20.800453 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:20.802502 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.802512 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.802529 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.803075 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.803387 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.803391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.803393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.803538 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.803960 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:20.803968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:20.804206 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 15:15:20.804394 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 15:15:20.804718 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":127,"threads_started":1}
I20260807 15:15:20.807078 11387 heartbeater.cc:347] Connected to a master server at 127.31.230.254:36387
I20260807 15:15:20.807219 11387 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:20.807818 11387 heartbeater.cc:511] Master 127.31.230.254:36387 requested a full tablet report, sending...
I20260807 15:15:20.809032 10907 ts_manager.cc:194] Registered new tserver with Master: 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337)
I20260807 15:15:20.809875 10907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:44691
WARNING: no policy specified for oryx/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:15:20.817544 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.809552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:55115 (local address 127.31.230.254:36387)
0807 15:15:20.809662 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:20.809666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:20.809981 (+ 315us) server_negotiation.cc:407] Connection header received
0807 15:15:20.810167 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:20.810173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:20.810224 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:20.810307 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:20.810843 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.811743 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.812623 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.813046 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.813185 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.814299 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:20.814304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:20.814308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:20.814337 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:20.815930 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.816366 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.816369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.816371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.816446 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:20.816690 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:20.816692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:20.816694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:20.816887 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:20.816966 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:20.817262 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:15:20.817369 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:15:20.817564 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.809440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.31.230.194:55115)
0807 15:15:20.809892 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.809909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:20.810054 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:20.810341 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:20.810344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:20.810578 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:20.810700 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.810708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.811893 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.811897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.812492 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:20.812501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:20.813167 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:20.813169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:20.813241 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:20.813812 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:20.813823 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:20.814137 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:20.816059 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.816066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.816068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.816263 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.816532 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:20.816534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:20.816535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:20.816610 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:20.817000 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:20.817004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:20.817086 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 15:15:20.817201 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:15:20.817362 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":242,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab.
Entry for principal oryx/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab.
I20260807 15:15:20.837327 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:36387
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:20.954471 11399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:20.954777 11399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:20.954892 11399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:20.959103 11399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:20.959189 11399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:20.959259 11399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:15:20.959342 11399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:20.964700 11399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.230.254:36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:20.965942 11399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:20.967033 11399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:20.977267 11404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:20.977280 11407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:20.977625 11399 server_base.cc:1034] running on GCE node
W20260807 15:15:20.977280 11405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:20.978194 11399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:20.978928 11399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:20.980115 11399 hybrid_clock.cc:648] HybridClock initialized: now 1786115720980100 us; error 27 us; skew 500 ppm
Aug 07 15:15:20 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:20.983258 11399 init.cc:377] Logged in from keytab as oryx/127.31.230.195@KRBTEST.COM (short username oryx)
I20260807 15:15:20.984283 11399 webserver.cc:460] Webserver started at http://127.31.230.195:40313/ using document root <none> and password file <none>
I20260807 15:15:20.984894 11399 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:20.984988 11399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:20.985226 11399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:20.987502 11399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "2ca88fb8f990457a89f24ee2442b1412"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "988cd939dc4c4e3f4be2b6264f09c585"
server_key_iv: "b2d7a793c81e0432f6c2eaaa6362dbca"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.988127 11399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "2ca88fb8f990457a89f24ee2442b1412"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "988cd939dc4c4e3f4be2b6264f09c585"
server_key_iv: "b2d7a793c81e0432f6c2eaaa6362dbca"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.992200 11399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 15:15:20.994992 11415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:20.996292 11399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:20.996441 11399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "2ca88fb8f990457a89f24ee2442b1412"
format_stamp: "Formatted at 2026-08-07 15:15:20 on dist-test-slave-gkw7"
server_key: "988cd939dc4c4e3f4be2b6264f09c585"
server_key_iv: "b2d7a793c81e0432f6c2eaaa6362dbca"
server_key_version: "encryptionkey@0"
I20260807 15:15:20.996569 11399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:21.030331 11399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:21.034039 11399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:21.034282 11399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:21.035885 11399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:21.035974 11399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:21.036051 11399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:21.036110 11399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:21.047034 11399 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:36621
I20260807 15:15:21.047091 11574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:36621 every 8 connection(s)
I20260807 15:15:21.048105 11399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:21.051436 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.036828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:60819 (local address 127.31.230.254:36387)
0807 15:15:21.036991 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.036996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.037971 (+ 975us) server_negotiation.cc:407] Connection header received
0807 15:15:21.039113 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.039116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.039183 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.039245 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.041196 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.042128 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.044001 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.044219 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.044313 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.047657 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:21.047662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:21.047664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:21.047698 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:21.049490 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.050026 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.050029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.050030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.050111 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.050417 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.050420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.050421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.050598 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:21.050676 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.051201 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.051310 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:15:21.052309 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.037284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.31.230.195:60819)
0807 15:15:21.037794 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.037847 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.038837 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.039716 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.039731 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.040241 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:21.041005 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.041029 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.042312 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.042317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.043869 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.043880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.044392 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.044398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.044641 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.045409 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:21.045437 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:21.047470 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:21.049653 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:21.049657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:21.049669 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:21.049933 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:21.050198 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:21.050200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:21.050202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:21.050336 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:21.050724 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:21.050729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:21.050966 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.051148 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.051425 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":143,"threads_started":1}
I20260807 15:15:21.053738 11576 heartbeater.cc:347] Connected to a master server at 127.31.230.254:36387
I20260807 15:15:21.053918 11576 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:21.054589 11576 heartbeater.cc:511] Master 127.31.230.254:36387 requested a full tablet report, sending...
I20260807 15:15:21.055325 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 11399
I20260807 15:15:21.055475 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:21.055799 32667 external_mini_cluster.cc:1428] Setting key b2a6f313f666641561c89c0c6523efaf
I20260807 15:15:21.055847 10911 ts_manager.cc:194] Registered new tserver with Master: 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.056602 10911 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:60819
I20260807 15:15:21.058053 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:21.065678 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.056072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.31.230.195:45773)
0807 15:15:21.057077 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.057095 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.057823 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.058148 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.058151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.058384 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:21.058505 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.058513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.059561 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.059564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.060123 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.060131 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.060799 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.060804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.060923 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.061524 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:21.061579 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:21.062092 (+ 513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:21.063875 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:21.063881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:21.063886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:21.064182 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:21.064538 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:21.064544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:21.064547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:21.064661 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:21.065137 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:21.065140 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:21.065229 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.065318 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.065437 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":115,"threads_started":1}
I20260807 15:15:21.065836 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.056132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:45773 (local address 127.31.230.254:36387)
0807 15:15:21.056261 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.056266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.057185 (+ 919us) server_negotiation.cc:407] Connection header received
0807 15:15:21.057954 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.057958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.058012 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.058109 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.058646 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.059416 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.060255 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.060652 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.060782 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.062245 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:21.062248 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:21.062250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:21.062271 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:21.063752 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.064339 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.064343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.064345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.064434 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.064773 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.064776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.064777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.065025 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:21.065136 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.065415 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.065585 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:15:21.087118 10973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:21.079944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.31.230.254:36387)
0807 15:15:21.080089 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.080093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.080107 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:21.080172 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.080174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.080216 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.080288 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.081123 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.081802 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.082698 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.082921 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.083005 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.086953 (+ 3948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39302: BlockingRecv error: recv got EOF from 127.0.0.1:39302 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:21.096585 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.088661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39310 (local address 127.31.230.254:36387)
0807 15:15:21.088801 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.088805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.088822 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:15:21.088955 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.088957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.088997 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.089068 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.089754 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.090269 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.091009 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.091226 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.091308 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.093619 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:21.093623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:21.093625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:21.093658 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:21.095145 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.095522 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.095525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.095526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.095602 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.095832 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.095834 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.095835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.096006 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:21.096111 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.096292 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.096388 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:15:21.099865 10911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:21.103322 10911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:21.119293 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.113450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.31.230.195:36621)
0807 15:15:21.114085 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.114095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.114162 (+ 67us) server_negotiation.cc:407] Connection header received
0807 15:15:21.114229 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.114233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.114400 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.114896 (+ 496us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.115053 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.116171 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.117578 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.118429 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.118604 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.118765 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.119005 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.119125 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":154,"threads_started":1}
I20260807 15:15:21.119333 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.112603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:33337 (local address 127.0.0.1:34174)
0807 15:15:21.113351 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.113404 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.113593 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.114462 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.114471 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.114502 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.114770 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.114781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.116186 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.116189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.117591 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.117603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.118593 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.118595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.118756 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.118827 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.118953 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.119144 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":246,"spinlock_wait_cycles":13312,"thread_start_us":84,"threads_started":1}
I20260807 15:15:21.119333 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.112810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36621 (local address 127.0.0.1:58940)
0807 15:15:21.113350 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.113405 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.113593 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.114540 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.114543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.114563 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.114741 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.114749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.116294 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.116298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.117419 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.117429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.118589 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.118593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.118760 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.118826 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.118953 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.119144 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":96,"spinlock_wait_cycles":12800,"thread_start_us":115,"threads_started":1}
I20260807 15:15:21.119848 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.112805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.31.230.194:33337)
0807 15:15:21.113919 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.113934 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.113963 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:15:21.114047 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.114052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.114277 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.114721 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.114873 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.116068 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.117726 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.118490 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.118642 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.118764 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.119578 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.119682 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":126,"threads_started":1}
I20260807 15:15:21.120275 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.113018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36233 (local address 127.0.0.1:58618)
0807 15:15:21.114108 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.114121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.114213 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.114943 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.114945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.114960 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.115018 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.115022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.115985 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.115991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.117591 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.117603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.119811 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.119816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.119939 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.119966 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.120030 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.120110 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":100,"thread_start_us":102,"threads_started":1}
I20260807 15:15:21.121074 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.113441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.31.230.193:36233)
0807 15:15:21.113970 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.113978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.114327 (+ 349us) server_negotiation.cc:407] Connection header received
0807 15:15:21.114417 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.114424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.114640 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.114787 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.115138 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.115862 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.118842 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.119680 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.119807 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.120789 (+ 982us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.120844 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.120946 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":128,"threads_started":1}
I20260807 15:15:21.122961 11289 tablet_service.cc:1467] Processing CreateTablet for tablet 70f1667ccb4540c7a3905d725145622c (DEFAULT_TABLE table=test-table [id=0fd89b2daa994b73816fb3024d84a229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.123014 11482 tablet_service.cc:1467] Processing CreateTablet for tablet 70f1667ccb4540c7a3905d725145622c (DEFAULT_TABLE table=test-table [id=0fd89b2daa994b73816fb3024d84a229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.124068 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f1667ccb4540c7a3905d725145622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.123764 11089 tablet_service.cc:1467] Processing CreateTablet for tablet 70f1667ccb4540c7a3905d725145622c (DEFAULT_TABLE table=test-table [id=0fd89b2daa994b73816fb3024d84a229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.124181 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f1667ccb4540c7a3905d725145622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.124716 11089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f1667ccb4540c7a3905d725145622c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.131881 11610 tablet_bootstrap.cc:492] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Bootstrap starting.
I20260807 15:15:21.133016 11611 tablet_bootstrap.cc:492] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Bootstrap starting.
I20260807 15:15:21.133900 11612 tablet_bootstrap.cc:492] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Bootstrap starting.
I20260807 15:15:21.134666 11610 tablet_bootstrap.cc:654] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.135205 11611 tablet_bootstrap.cc:654] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.135542 11610 log.cc:824] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:21.135946 11612 tablet_bootstrap.cc:654] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.136046 11611 log.cc:824] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:21.136840 11612 log.cc:824] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:21.138214 11610 tablet_bootstrap.cc:492] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: No bootstrap required, opened a new log
I20260807 15:15:21.138273 11611 tablet_bootstrap.cc:492] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: No bootstrap required, opened a new log
I20260807 15:15:21.138463 11610 ts_tablet_manager.cc:1397] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:15:21.138465 11611 ts_tablet_manager.cc:1397] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:15:21.138885 11612 tablet_bootstrap.cc:492] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: No bootstrap required, opened a new log
I20260807 15:15:21.139067 11612 ts_tablet_manager.cc:1397] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:15:21.141489 11610 raft_consensus.cc:348] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.141841 11610 raft_consensus.cc:374] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.141927 11610 raft_consensus.cc:729] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.142406 11610 consensus_queue.cc:260] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.143172 11611 raft_consensus.cc:348] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.143172 11612 raft_consensus.cc:348] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.143440 11612 raft_consensus.cc:374] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.143472 11611 raft_consensus.cc:374] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.143497 11610 ts_tablet_manager.cc:1428] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:15:21.143525 11612 raft_consensus.cc:729] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca88fb8f990457a89f24ee2442b1412, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.143548 11611 raft_consensus.cc:729] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c8544156d34bcf88e1e3345f34658f, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.143963 11387 heartbeater.cc:503] Master 127.31.230.254:36387 was elected leader, sending a full tablet report...
I20260807 15:15:21.144016 11611 consensus_queue.cc:260] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.144016 11612 consensus_queue.cc:260] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.144985 11612 ts_tablet_manager.cc:1428] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:15:21.144985 11611 ts_tablet_manager.cc:1428] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:15:21.145078 11177 heartbeater.cc:503] Master 127.31.230.254:36387 was elected leader, sending a full tablet report...
I20260807 15:15:21.145375 11576 heartbeater.cc:503] Master 127.31.230.254:36387 was elected leader, sending a full tablet report...
I20260807 15:15:21.278049 11618 raft_consensus.cc:482] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:21.278340 11618 raft_consensus.cc:504] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.279896 11618 leader_election.cc:288] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621), 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233)
I20260807 15:15:21.284888 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.280433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:35325 (local address 127.31.230.195:36621)
0807 15:15:21.280747 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.280752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.280771 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:15:21.280835 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.280838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.280897 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.281008 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.281498 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.282332 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.283721 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.284231 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.284390 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.284460 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.284663 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.284745 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260807 15:15:21.284914 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.280111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36621 (local address 127.31.230.194:35325)
0807 15:15:21.280249 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.280268 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.280384 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.281186 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.281190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.281209 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.281357 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.281363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.282453 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.282457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.283564 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.283575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.284381 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.284385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.284507 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.284533 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.284645 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.284743 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":39}
I20260807 15:15:21.285090 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.280734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:58365 (local address 127.31.230.193:36233)
0807 15:15:21.280862 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.280867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.280880 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:21.280935 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.280939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.280997 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.281080 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.281447 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.282089 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.283686 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.284372 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.284488 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.284536 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.284863 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.284953 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:15:21.285458 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.280203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36233 (local address 127.31.230.194:58365)
0807 15:15:21.280637 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.280651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.280759 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.281106 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.281109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.281128 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.281311 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.281319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.282222 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.282227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.283526 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.283540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.284598 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.284602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.284707 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.284729 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.285227 (+ 498us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.285309 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":4,"thread_start_us":151,"threads_started":1}
I20260807 15:15:21.285730 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1667ccb4540c7a3905d725145622c" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412" is_pre_election: true
I20260807 15:15:21.286121 11519 raft_consensus.cc:2454] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.286294 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1667ccb4540c7a3905d725145622c" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f" is_pre_election: true
I20260807 15:15:21.286597 11111 raft_consensus.cc:2454] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.286705 11223 leader_election.cc:302] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.287340 11618 raft_consensus.cc:2781] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:21.287480 11618 raft_consensus.cc:482] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:21.287549 11618 raft_consensus.cc:3037] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.289222 11618 raft_consensus.cc:504] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.289691 11618 leader_election.cc:288] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Requested vote from peers 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621), 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233)
I20260807 15:15:21.290062 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1667ccb4540c7a3905d725145622c" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412"
I20260807 15:15:21.290112 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1667ccb4540c7a3905d725145622c" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f"
I20260807 15:15:21.290223 11519 raft_consensus.cc:3037] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.290253 11111 raft_consensus.cc:3037] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.291491 11519 raft_consensus.cc:2454] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.291558 11111 raft_consensus.cc:2454] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.291785 11223 leader_election.cc:302] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.292138 11618 raft_consensus.cc:2781] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:21.292423 11618 raft_consensus.cc:686] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Becoming Leader. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Running, Role: LEADER
I20260807 15:15:21.292758 11618 consensus_queue.cc:237] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } }
I20260807 15:15:21.296072 10911 catalog_manager.cc:5495] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512b8289b81f499ea3023798555c1af0 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "512b8289b81f499ea3023798555c1af0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } health_report { overall_health: UNKNOWN } } }
W20260807 15:15:21.299306 11577 tablet.cc:2369] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:21.303372 11388 tablet.cc:2369] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:21.311232 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.307344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34186 (local address 127.31.230.194:33337)
0807 15:15:21.307478 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.307484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.307524 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:15:21.307731 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.307734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.307787 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.307901 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:21.308194 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.308809 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.309767 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.310002 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.310119 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.310411 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.310838 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.310969 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.311035 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.311094 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:15:21.316890 11519 raft_consensus.cc:1261] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.316891 11111 raft_consensus.cc:1261] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.317617 11618 consensus_queue.cc:1035] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.317888 11624 consensus_queue.cc:1035] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.326606 11633 mvcc.cc:205] Tried to move back new op lower bound from 7315929994506969088 to 7315929994418024448. Current Snapshot: MvccSnapshot[applied={T|T < 7315929994506969088}]
I20260807 15:15:21.328048 11629 mvcc.cc:205] Tried to move back new op lower bound from 7315929994506969088 to 7315929994418024448. Current Snapshot: MvccSnapshot[applied={T|T < 7315929994506969088}]
I20260807 15:15:21.333662 10907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:21.333813 10907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:21.336239 10907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Sending DeleteTablet for 3 replicas of tablet 70f1667ccb4540c7a3905d725145622c
I20260807 15:15:21.337067 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 70f1667ccb4540c7a3905d725145622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34174
I20260807 15:15:21.337142 11482 tablet_service.cc:1514] Processing DeleteTablet for tablet 70f1667ccb4540c7a3905d725145622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58940
I20260807 15:15:21.337862 11646 tablet_replica.cc:330] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: stopping tablet replica
I20260807 15:15:21.337970 11089 tablet_service.cc:1514] Processing DeleteTablet for tablet 70f1667ccb4540c7a3905d725145622c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58618
I20260807 15:15:21.338202 11646 raft_consensus.cc:2229] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:21.338586 11646 raft_consensus.cc:2258] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.338625 11647 tablet_replica.cc:330] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: stopping tablet replica
I20260807 15:15:21.338608 10907 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:15:21.338912 11647 raft_consensus.cc:2229] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 15:15:21.339056 10907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:21.339239 11647 raft_consensus.cc:2258] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.340678 11647 ts_tablet_manager.cc:1905] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.341671 11646 ts_tablet_manager.cc:1905] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.343819 11647 ts_tablet_manager.cc:1918] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:21.343945 11647 log.cc:1196] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/70f1667ccb4540c7a3905d725145622c
I20260807 15:15:21.344362 11647 ts_tablet_manager.cc:1939] T 70f1667ccb4540c7a3905d725145622c P 93c8544156d34bcf88e1e3345f34658f: Deleting consensus metadata
I20260807 15:15:21.344780 11482 tablet_service.cc:1467] Processing CreateTablet for tablet c5a90e2c64d041c1a2418b2532adabef (DEFAULT_TABLE table=test-table [id=931e662f1c39470ab987da38f94a0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.344956 11289 tablet_service.cc:1467] Processing CreateTablet for tablet c5a90e2c64d041c1a2418b2532adabef (DEFAULT_TABLE table=test-table [id=931e662f1c39470ab987da38f94a0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.345070 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a90e2c64d041c1a2418b2532adabef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.345713 11089 tablet_service.cc:1467] Processing CreateTablet for tablet c5a90e2c64d041c1a2418b2532adabef (DEFAULT_TABLE table=test-table [id=931e662f1c39470ab987da38f94a0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.345767 10883 catalog_manager.cc:4841] TS 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233): tablet 70f1667ccb4540c7a3905d725145622c (table test-table [id=0fd89b2daa994b73816fb3024d84a229]) successfully deleted
I20260807 15:15:21.345960 11089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a90e2c64d041c1a2418b2532adabef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.347373 11612 tablet_bootstrap.cc:492] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Bootstrap starting.
I20260807 15:15:21.348060 11611 tablet_bootstrap.cc:492] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Bootstrap starting.
I20260807 15:15:21.348337 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a90e2c64d041c1a2418b2532adabef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.348376 11612 tablet_bootstrap.cc:654] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.349066 11611 tablet_bootstrap.cc:654] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.350168 11612 tablet_bootstrap.cc:492] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: No bootstrap required, opened a new log
I20260807 15:15:21.350274 11612 ts_tablet_manager.cc:1397] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:21.350764 11610 tablet_bootstrap.cc:492] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Bootstrap starting.
I20260807 15:15:21.350814 11612 raft_consensus.cc:348] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.350916 11612 raft_consensus.cc:374] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.350967 11612 raft_consensus.cc:729] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca88fb8f990457a89f24ee2442b1412, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.351094 11612 consensus_queue.cc:260] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.351383 11612 ts_tablet_manager.cc:1428] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.351748 11610 tablet_bootstrap.cc:654] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.353143 11643 tablet_replica.cc:330] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: stopping tablet replica
I20260807 15:15:21.353232 11610 tablet_bootstrap.cc:492] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: No bootstrap required, opened a new log
I20260807 15:15:21.353317 11610 ts_tablet_manager.cc:1397] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:21.353718 11643 raft_consensus.cc:2229] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:21.353804 11610 raft_consensus.cc:348] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.353911 11610 raft_consensus.cc:374] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.353952 11610 raft_consensus.cc:729] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.354038 11643 raft_consensus.cc:2258] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.354096 11610 consensus_queue.cc:260] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.354380 11610 ts_tablet_manager.cc:1428] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.355011 11611 tablet_bootstrap.cc:492] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: No bootstrap required, opened a new log
I20260807 15:15:21.355137 11611 ts_tablet_manager.cc:1397] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 15:15:21.355934 11611 raft_consensus.cc:348] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.356074 11611 raft_consensus.cc:374] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.356130 11611 raft_consensus.cc:729] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c8544156d34bcf88e1e3345f34658f, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.356273 11611 consensus_queue.cc:260] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.356539 11611 ts_tablet_manager.cc:1428] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:21.356678 11646 ts_tablet_manager.cc:1918] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:21.356768 11646 log.cc:1196] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/70f1667ccb4540c7a3905d725145622c
I20260807 15:15:21.357098 11646 ts_tablet_manager.cc:1939] T 70f1667ccb4540c7a3905d725145622c P 512b8289b81f499ea3023798555c1af0: Deleting consensus metadata
I20260807 15:15:21.357120 11640 raft_consensus.cc:482] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:21.357204 11640 raft_consensus.cc:504] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.357378 11643 ts_tablet_manager.cc:1905] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.357658 11640 leader_election.cc:288] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.357977 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a90e2c64d041c1a2418b2532adabef" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412" is_pre_election: true
I20260807 15:15:21.358094 11519 raft_consensus.cc:2454] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.358139 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a90e2c64d041c1a2418b2532adabef" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f" is_pre_election: true
I20260807 15:15:21.358266 11111 raft_consensus.cc:2454] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.358414 11223 leader_election.cc:302] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.358587 11640 raft_consensus.cc:2781] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:21.358635 11640 raft_consensus.cc:482] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:21.358672 11640 raft_consensus.cc:3037] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.359714 11640 raft_consensus.cc:504] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.360044 11640 leader_election.cc:288] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Requested vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.360373 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a90e2c64d041c1a2418b2532adabef" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f"
I20260807 15:15:21.360421 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a90e2c64d041c1a2418b2532adabef" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412"
I20260807 15:15:21.360483 11111 raft_consensus.cc:3037] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.360520 11519 raft_consensus.cc:3037] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.361586 11519 raft_consensus.cc:2454] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.361712 11111 raft_consensus.cc:2454] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.361891 11223 leader_election.cc:302] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.362100 11640 raft_consensus.cc:2781] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:21.362175 11640 raft_consensus.cc:686] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Becoming Leader. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Running, Role: LEADER
I20260807 15:15:21.362287 11640 consensus_queue.cc:237] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.363878 10906 catalog_manager.cc:5495] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512b8289b81f499ea3023798555c1af0 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "512b8289b81f499ea3023798555c1af0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:21.364125 10885 catalog_manager.cc:4841] TS 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337): tablet 70f1667ccb4540c7a3905d725145622c (table test-table [id=0fd89b2daa994b73816fb3024d84a229]) successfully deleted
I20260807 15:15:21.366291 11643 ts_tablet_manager.cc:1918] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:21.366453 11643 log.cc:1196] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/70f1667ccb4540c7a3905d725145622c
I20260807 15:15:21.366844 11643 ts_tablet_manager.cc:1939] T 70f1667ccb4540c7a3905d725145622c P 2ca88fb8f990457a89f24ee2442b1412: Deleting consensus metadata
I20260807 15:15:21.368018 10889 catalog_manager.cc:4841] TS 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621): tablet 70f1667ccb4540c7a3905d725145622c (table test-table [id=0fd89b2daa994b73816fb3024d84a229]) successfully deleted
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115720, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.254@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:21.413050 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.403654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.31.230.254:36387)
0807 15:15:21.403810 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.403816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.404002 (+ 186us) server_negotiation.cc:407] Connection header received
0807 15:15:21.404125 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.404129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.404184 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.404302 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.404987 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.405872 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.406746 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.407095 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.407200 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.409825 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:21.409828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:21.409830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:21.409851 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:21.411206 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.411776 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.411778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.411780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.411852 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.412133 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.412135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.412136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.412308 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:21.412384 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.412737 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.412902 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":60}
I20260807 15:15:21.413060 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.403545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:36387 (local address 127.0.0.1:39320)
0807 15:15:21.403900 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.403924 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.404033 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.404334 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.404337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.404731 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:21.404848 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.404855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.405976 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.405979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.406632 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.406643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.407256 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.407258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.407324 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.407962 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:21.407974 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:21.409678 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:21.411317 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:21.411327 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:21.411336 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:21.411662 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:21.411940 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:21.411943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:21.411945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:21.412044 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:21.412411 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:21.412418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:21.412560 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.412685 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.412845 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":133,"threads_started":1}
I20260807 15:15:21.417253 10906 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:39320:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:15:21.418114 10906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:21.423151 11089 tablet_service.cc:1467] Processing CreateTablet for tablet d4dfcd24ca5a4c7f82a793bdfac15eb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=abe66a4e5c154bab83e61cf073ea1497]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:15:21.423175 11289 tablet_service.cc:1467] Processing CreateTablet for tablet d4dfcd24ca5a4c7f82a793bdfac15eb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=abe66a4e5c154bab83e61cf073ea1497]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:15:21.423369 11482 tablet_service.cc:1467] Processing CreateTablet for tablet d4dfcd24ca5a4c7f82a793bdfac15eb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=abe66a4e5c154bab83e61cf073ea1497]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:15:21.423589 11089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4dfcd24ca5a4c7f82a793bdfac15eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.423605 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4dfcd24ca5a4c7f82a793bdfac15eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.423709 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4dfcd24ca5a4c7f82a793bdfac15eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.426575 11612 tablet_bootstrap.cc:492] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: Bootstrap starting.
I20260807 15:15:21.426678 11611 tablet_bootstrap.cc:492] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: Bootstrap starting.
I20260807 15:15:21.427618 11612 tablet_bootstrap.cc:654] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.427618 11611 tablet_bootstrap.cc:654] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.428691 11610 tablet_bootstrap.cc:492] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: Bootstrap starting.
I20260807 15:15:21.428972 11611 tablet_bootstrap.cc:492] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: No bootstrap required, opened a new log
I20260807 15:15:21.429045 11611 ts_tablet_manager.cc:1397] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:21.429463 11611 raft_consensus.cc:348] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.429644 11611 raft_consensus.cc:374] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.429678 11611 raft_consensus.cc:729] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c8544156d34bcf88e1e3345f34658f, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.429692 11610 tablet_bootstrap.cc:654] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.429804 11611 consensus_queue.cc:260] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.430078 11621 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.430235 11621 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:15:21.430074 11611 ts_tablet_manager.cc:1428] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.430860 11612 tablet_bootstrap.cc:492] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: No bootstrap required, opened a new log
I20260807 15:15:21.430976 11612 ts_tablet_manager.cc:1397] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:15:21.431309 11610 tablet_bootstrap.cc:492] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: No bootstrap required, opened a new log
I20260807 15:15:21.431382 11610 ts_tablet_manager.cc:1397] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:15:21.431341 11612 raft_consensus.cc:348] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.431423 11612 raft_consensus.cc:374] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.431444 11612 raft_consensus.cc:729] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca88fb8f990457a89f24ee2442b1412, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.431586 11612 consensus_queue.cc:260] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.431797 11610 raft_consensus.cc:348] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.431898 11610 raft_consensus.cc:374] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.431828 11622 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.431929 11610 raft_consensus.cc:729] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.432040 11610 consensus_queue.cc:260] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.431880 11612 ts_tablet_manager.cc:1428] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:21.432288 11610 ts_tablet_manager.cc:1428] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:21.432472 11618 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.432600 11618 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:15:21.431994 11622 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:15:21.520578 11622 raft_consensus.cc:482] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:21.520761 11622 raft_consensus.cc:504] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.522063 11622 leader_election.cc:288] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337)
I20260807 15:15:21.526361 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.522307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36233 (local address 127.31.230.195:34861)
0807 15:15:21.522507 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.522523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.522641 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.522947 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.522950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.522964 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.523111 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.523118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.523926 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.523929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.525046 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.525057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.525905 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.525909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.526022 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.526043 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.526145 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.526224 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 15:15:21.526361 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.522395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:34861 (local address 127.31.230.193:36233)
0807 15:15:21.522530 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.522536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.522643 (+ 107us) server_negotiation.cc:407] Connection header received
0807 15:15:21.522749 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.522752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.522797 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.522912 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.523234 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.523819 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.525177 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.525771 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.525904 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.525960 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.526163 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.526236 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:15:21.526583 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.522510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:42229 (local address 127.31.230.194:33337)
0807 15:15:21.522661 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.522668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.522843 (+ 175us) server_negotiation.cc:407] Connection header received
0807 15:15:21.522904 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.522907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.522955 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.523041 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:21.523711 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.524303 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.525613 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.526019 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.526163 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.526215 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.526408 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.526481 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:15:21.526599 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.522373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:33337 (local address 127.31.230.195:42229)
0807 15:15:21.522731 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.522744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.522839 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.523061 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.523064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.523083 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:21.523581 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.523590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.524436 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.524439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.525459 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.525467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.526154 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.526157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.526273 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.526292 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.526375 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.526450 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":83,"threads_started":1}
I20260807 15:15:21.527061 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4dfcd24ca5a4c7f82a793bdfac15eb8" candidate_uuid: "2ca88fb8f990457a89f24ee2442b1412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f" is_pre_election: true
I20260807 15:15:21.527240 11111 raft_consensus.cc:2454] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ca88fb8f990457a89f24ee2442b1412 in term 0.
I20260807 15:15:21.527237 11317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4dfcd24ca5a4c7f82a793bdfac15eb8" candidate_uuid: "2ca88fb8f990457a89f24ee2442b1412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b8289b81f499ea3023798555c1af0" is_pre_election: true
I20260807 15:15:21.527513 11317 raft_consensus.cc:2454] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ca88fb8f990457a89f24ee2442b1412 in term 0.
I20260807 15:15:21.527658 11416 leader_election.cc:302] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 93c8544156d34bcf88e1e3345f34658f; no voters:
I20260807 15:15:21.527941 11622 raft_consensus.cc:2781] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:21.528010 11622 raft_consensus.cc:482] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:21.528049 11622 raft_consensus.cc:3037] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.529201 11622 raft_consensus.cc:504] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.529800 11622 leader_election.cc:288] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [CANDIDATE]: Term 1 election: Requested vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337)
I20260807 15:15:21.530205 11317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4dfcd24ca5a4c7f82a793bdfac15eb8" candidate_uuid: "2ca88fb8f990457a89f24ee2442b1412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b8289b81f499ea3023798555c1af0"
I20260807 15:15:21.530207 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4dfcd24ca5a4c7f82a793bdfac15eb8" candidate_uuid: "2ca88fb8f990457a89f24ee2442b1412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f"
I20260807 15:15:21.530354 11317 raft_consensus.cc:3037] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.530364 11111 raft_consensus.cc:3037] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.531318 11111 raft_consensus.cc:2454] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ca88fb8f990457a89f24ee2442b1412 in term 1.
I20260807 15:15:21.531526 11317 raft_consensus.cc:2454] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ca88fb8f990457a89f24ee2442b1412 in term 1.
I20260807 15:15:21.531689 11416 leader_election.cc:302] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 93c8544156d34bcf88e1e3345f34658f; no voters:
I20260807 15:15:21.531911 11622 raft_consensus.cc:2781] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:21.532156 11622 raft_consensus.cc:686] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 LEADER]: Becoming Leader. State: Replica: 2ca88fb8f990457a89f24ee2442b1412, State: Running, Role: LEADER
I20260807 15:15:21.532447 11622 consensus_queue.cc:237] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.533820 11669 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: TxnStatusTablet state changed. Reason: New leader 2ca88fb8f990457a89f24ee2442b1412. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.533946 11669 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:15:21.534251 11671 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:15:21.534427 11671 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:15:21.535292 10906 catalog_manager.cc:5495] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195). New cstate: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } health_report { overall_health: HEALTHY } } }
I20260807 15:15:21.563778 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.559650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.31.230.195:36621)
0807 15:15:21.559787 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.559792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.559838 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:15:21.560058 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.560060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.560117 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.560233 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:21.560517 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.561029 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.562301 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.562516 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.562600 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.562949 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.563396 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.563487 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.563605 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.563658 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:15:21.563817 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.559525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36621 (local address 127.0.0.1:58952)
0807 15:15:21.559775 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.559789 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.559909 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.560256 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.560259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.560266 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:15:21.560402 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.560409 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.561137 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.561141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.562185 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.562198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.562629 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.562633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.562735 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.562797 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.563495 (+ 698us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.563510 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.563584 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.563673 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260807 15:15:21.567164 11111 raft_consensus.cc:1261] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Refusing update from remote peer 2ca88fb8f990457a89f24ee2442b1412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.567360 11317 raft_consensus.cc:1261] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Refusing update from remote peer 2ca88fb8f990457a89f24ee2442b1412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.567745 11669 consensus_queue.cc:1035] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.567905 11622 consensus_queue.cc:1035] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.569644 11621 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: TxnStatusTablet state changed. Reason: New leader 2ca88fb8f990457a89f24ee2442b1412. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.569787 11621 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:15:21.569870 11618 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: TxnStatusTablet state changed. Reason: New leader 2ca88fb8f990457a89f24ee2442b1412. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.569996 11618 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:15:21.571925 11622 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.572041 11622 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:15:21.572176 11622 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.572273 11622 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 2ca88fb8f990457a89f24ee2442b1412: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:15:21.572290 11621 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.572386 11621 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 93c8544156d34bcf88e1e3345f34658f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:15:21.572944 11627 mvcc.cc:205] Tried to move back new op lower bound from 7315929995533398016 to 7315929995399716864. Current Snapshot: MvccSnapshot[applied={T|T < 7315929995533398016}]
I20260807 15:15:21.573096 11618 tablet_replica.cc:439] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ca88fb8f990457a89f24ee2442b1412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } } }
I20260807 15:15:21.573204 11618 tablet_replica.cc:442] T d4dfcd24ca5a4c7f82a793bdfac15eb8 P 512b8289b81f499ea3023798555c1af0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 15:15:21.577594 11179 tablet.cc:2369] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:21.593278 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.588845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36621 (local address 127.31.230.194:43617)
0807 15:15:21.589208 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.589226 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.589336 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.589691 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.589694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.589703 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:15:21.589831 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.589838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.590858 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.590862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.591863 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.591872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.592499 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.592502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.592596 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.592650 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.592986 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.592994 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.593067 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.593153 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":139,"threads_started":1}
I20260807 15:15:21.593289 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.589282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:43617 (local address 127.31.230.195:36621)
0807 15:15:21.589427 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.589432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.589447 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:21.589502 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.589505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.589581 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.589663 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:21.589953 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.590743 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.591994 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.592376 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.592484 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.592795 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.592885 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.592959 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.593091 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.593145 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:15:21.600575 11519 raft_consensus.cc:1261] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.600577 11111 raft_consensus.cc:1261] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.600991 11618 consensus_queue.cc:1035] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.601200 11640 consensus_queue.cc:1035] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.623770 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.619663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:49505 (local address 127.31.230.194:33337)
0807 15:15:21.619783 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.619789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.620185 (+ 396us) server_negotiation.cc:407] Connection header received
0807 15:15:21.620343 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.620346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.620401 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.620480 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:21.620939 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.621661 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.622589 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.622829 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.622945 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.623274 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.623353 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.623438 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.623572 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.623626 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:15:21.623802 11692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.619600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:33337 (local address 127.31.230.195:49505)
0807 15:15:21.620066 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.620106 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:21.620233 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:21.620647 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:21.620650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:21.620659 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:15:21.620833 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.620843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.621783 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.621790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.622461 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:21.622470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.622970 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.622974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:21.623082 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.623125 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.623482 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.623491 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:15:21.623559 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:15:21.623638 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":128,"threads_started":1}
I20260807 15:15:21.644964 10911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:21.645140 10911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:21.647336 10911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Sending DeleteTablet for 3 replicas of tablet c5a90e2c64d041c1a2418b2532adabef
I20260807 15:15:21.647902 11089 tablet_service.cc:1514] Processing DeleteTablet for tablet c5a90e2c64d041c1a2418b2532adabef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58618
I20260807 15:15:21.647923 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet c5a90e2c64d041c1a2418b2532adabef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34174
I20260807 15:15:21.648089 11647 tablet_replica.cc:330] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: stopping tablet replica
I20260807 15:15:21.648087 11482 tablet_service.cc:1514] Processing DeleteTablet for tablet c5a90e2c64d041c1a2418b2532adabef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58940
I20260807 15:15:21.648190 11647 raft_consensus.cc:2229] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:21.648267 11643 tablet_replica.cc:330] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: stopping tablet replica
I20260807 15:15:21.648309 11647 raft_consensus.cc:2258] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.648375 11643 raft_consensus.cc:2229] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:21.648464 11646 tablet_replica.cc:330] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: stopping tablet replica
I20260807 15:15:21.648479 11643 raft_consensus.cc:2258] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.648602 11646 raft_consensus.cc:2229] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:21.648790 11646 raft_consensus.cc:2258] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.649776 11646 ts_tablet_manager.cc:1905] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.652166 11647 ts_tablet_manager.cc:1905] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.652904 11646 ts_tablet_manager.cc:1918] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:15:21.653040 11646 log.cc:1196] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/c5a90e2c64d041c1a2418b2532adabef
I20260807 15:15:21.653376 11646 ts_tablet_manager.cc:1939] T c5a90e2c64d041c1a2418b2532adabef P 512b8289b81f499ea3023798555c1af0: Deleting consensus metadata
I20260807 15:15:21.654280 10885 catalog_manager.cc:4841] TS 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337): tablet c5a90e2c64d041c1a2418b2532adabef (table test-table [id=931e662f1c39470ab987da38f94a0169]) successfully deleted
I20260807 15:15:21.654989 11643 ts_tablet_manager.cc:1905] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.655061 11647 ts_tablet_manager.cc:1918] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:15:21.655130 11647 log.cc:1196] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/c5a90e2c64d041c1a2418b2532adabef
I20260807 15:15:21.655392 11647 ts_tablet_manager.cc:1939] T c5a90e2c64d041c1a2418b2532adabef P 93c8544156d34bcf88e1e3345f34658f: Deleting consensus metadata
I20260807 15:15:21.656080 10883 catalog_manager.cc:4841] TS 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233): tablet c5a90e2c64d041c1a2418b2532adabef (table test-table [id=931e662f1c39470ab987da38f94a0169]) successfully deleted
I20260807 15:15:21.657763 11643 ts_tablet_manager.cc:1918] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:15:21.657819 11643 log.cc:1196] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/c5a90e2c64d041c1a2418b2532adabef
I20260807 15:15:21.658000 11643 ts_tablet_manager.cc:1939] T c5a90e2c64d041c1a2418b2532adabef P 2ca88fb8f990457a89f24ee2442b1412: Deleting consensus metadata
I20260807 15:15:21.658922 10889 catalog_manager.cc:4841] TS 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621): tablet c5a90e2c64d041c1a2418b2532adabef (table test-table [id=931e662f1c39470ab987da38f94a0169]) successfully deleted
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:15:21.681507 10973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:21.674239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.31.230.254:36387)
0807 15:15:21.674376 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.674379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.674643 (+ 264us) server_negotiation.cc:407] Connection header received
0807 15:15:21.674816 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.674821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.674880 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.674975 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.675783 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.676328 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.677256 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.677485 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.677665 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.681374 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39334: BlockingRecv error: recv got EOF from 127.0.0.1:39334 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 15:15:21 dist-test-slave-gkw7 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM
I20260807 15:15:21.692227 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.683478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.31.230.254:36387)
0807 15:15:21.683590 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.683595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.683702 (+ 107us) server_negotiation.cc:407] Connection header received
0807 15:15:21.683851 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.683861 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.683902 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.683988 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:21.684681 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.685184 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.686037 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.686261 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.686374 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.688749 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:21.688752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:21.688754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:21.688779 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:21.690484 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.690951 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.690955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.690956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.691036 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:21.691325 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:21.691328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:21.691330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:21.691524 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:21.691666 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.691918 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.692039 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:15:21.695554 10911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:21.696069 10911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:21.701471 11289 tablet_service.cc:1467] Processing CreateTablet for tablet f5e760980f274f5db9eb3d92a018cfab (DEFAULT_TABLE table=test-table [id=4b8179d84e2a4b4caf3e4a298d8f52b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.701436 11089 tablet_service.cc:1467] Processing CreateTablet for tablet f5e760980f274f5db9eb3d92a018cfab (DEFAULT_TABLE table=test-table [id=4b8179d84e2a4b4caf3e4a298d8f52b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.701884 11089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e760980f274f5db9eb3d92a018cfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.701892 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e760980f274f5db9eb3d92a018cfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.703881 11610 tablet_bootstrap.cc:492] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Bootstrap starting.
I20260807 15:15:21.704433 11611 tablet_bootstrap.cc:492] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Bootstrap starting.
I20260807 15:15:21.704756 11610 tablet_bootstrap.cc:654] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.705703 11611 tablet_bootstrap.cc:654] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.705961 11610 tablet_bootstrap.cc:492] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: No bootstrap required, opened a new log
I20260807 15:15:21.706045 11610 ts_tablet_manager.cc:1397] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 15:15:21.706270 11482 tablet_service.cc:1467] Processing CreateTablet for tablet f5e760980f274f5db9eb3d92a018cfab (DEFAULT_TABLE table=test-table [id=4b8179d84e2a4b4caf3e4a298d8f52b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.706460 11610 raft_consensus.cc:348] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.706566 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e760980f274f5db9eb3d92a018cfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.706576 11610 raft_consensus.cc:374] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.706614 11610 raft_consensus.cc:729] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.706741 11610 consensus_queue.cc:260] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.707036 11610 ts_tablet_manager.cc:1428] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.707582 11611 tablet_bootstrap.cc:492] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: No bootstrap required, opened a new log
I20260807 15:15:21.707657 11611 ts_tablet_manager.cc:1397] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:21.708252 11611 raft_consensus.cc:348] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.708356 11611 raft_consensus.cc:374] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.708384 11611 raft_consensus.cc:729] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c8544156d34bcf88e1e3345f34658f, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.708510 11611 consensus_queue.cc:260] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.708742 11611 ts_tablet_manager.cc:1428] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.709198 11612 tablet_bootstrap.cc:492] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Bootstrap starting.
I20260807 15:15:21.710163 11612 tablet_bootstrap.cc:654] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.712087 11612 tablet_bootstrap.cc:492] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: No bootstrap required, opened a new log
I20260807 15:15:21.712196 11612 ts_tablet_manager.cc:1397] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:21.712595 11612 raft_consensus.cc:348] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.712713 11612 raft_consensus.cc:374] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.712755 11612 raft_consensus.cc:729] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca88fb8f990457a89f24ee2442b1412, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.712884 11612 consensus_queue.cc:260] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.713165 11612 ts_tablet_manager.cc:1428] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.831357 11618 raft_consensus.cc:482] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:21.831512 11618 raft_consensus.cc:504] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.832086 11618 leader_election.cc:288] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.832612 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e760980f274f5db9eb3d92a018cfab" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f" is_pre_election: true
I20260807 15:15:21.832597 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e760980f274f5db9eb3d92a018cfab" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412" is_pre_election: true
I20260807 15:15:21.832773 11111 raft_consensus.cc:2454] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.832773 11519 raft_consensus.cc:2454] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.833101 11223 leader_election.cc:302] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.833292 11618 raft_consensus.cc:2781] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:21.833374 11618 raft_consensus.cc:482] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:21.833407 11618 raft_consensus.cc:3037] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.834548 11618 raft_consensus.cc:504] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.834880 11618 leader_election.cc:288] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Requested vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.835248 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e760980f274f5db9eb3d92a018cfab" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f"
I20260807 15:15:21.835328 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e760980f274f5db9eb3d92a018cfab" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412"
I20260807 15:15:21.835431 11111 raft_consensus.cc:3037] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.835479 11519 raft_consensus.cc:3037] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.836557 11111 raft_consensus.cc:2454] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.836580 11519 raft_consensus.cc:2454] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.836920 11223 leader_election.cc:302] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.837127 11618 raft_consensus.cc:2781] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:21.837258 11618 raft_consensus.cc:686] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Becoming Leader. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Running, Role: LEADER
I20260807 15:15:21.837419 11618 consensus_queue.cc:237] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.839234 10904 catalog_manager.cc:5495] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512b8289b81f499ea3023798555c1af0 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "512b8289b81f499ea3023798555c1af0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:21.866750 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.863432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.31.230.194:33337)
0807 15:15:21.863592 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:21.863597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:21.863610 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:21.863659 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:21.863661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:21.863707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:21.863773 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:21.864021 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.864565 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.865420 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:21.865687 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:21.865844 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:21.866187 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:21.866268 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:21.866374 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:21.866518 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:15:21.866583 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:15:21.868990 11519 raft_consensus.cc:1261] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.869048 11111 raft_consensus.cc:1261] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:21.869493 11624 consensus_queue.cc:1035] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.869746 11618 consensus_queue.cc:1035] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:21.877569 10911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:21.877724 10911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:21.879933 10911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Sending DeleteTablet for 3 replicas of tablet f5e760980f274f5db9eb3d92a018cfab
I20260807 15:15:21.880681 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e760980f274f5db9eb3d92a018cfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34174
I20260807 15:15:21.880697 11089 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e760980f274f5db9eb3d92a018cfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58618
I20260807 15:15:21.880860 11646 tablet_replica.cc:330] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: stopping tablet replica
I20260807 15:15:21.880833 11482 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e760980f274f5db9eb3d92a018cfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58940
I20260807 15:15:21.880899 11647 tablet_replica.cc:330] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: stopping tablet replica
I20260807 15:15:21.880936 11088 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e760980f274f5db9eb3d92a018cfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:21 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58618
I20260807 15:15:21.880975 11646 raft_consensus.cc:2229] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:21.881016 11647 raft_consensus.cc:2229] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:21.881143 11647 raft_consensus.cc:2258] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.881127 11646 raft_consensus.cc:2258] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.881327 11643 tablet_replica.cc:330] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: stopping tablet replica
I20260807 15:15:21.881433 11643 raft_consensus.cc:2229] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:21.881574 11643 raft_consensus.cc:2258] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:21.881858 11647 ts_tablet_manager.cc:1905] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.881927 10911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:21.882329 10911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:21.882428 11646 ts_tablet_manager.cc:1905] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:21.884644 11647 ts_tablet_manager.cc:1918] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:21.884727 11647 log.cc:1196] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/f5e760980f274f5db9eb3d92a018cfab
I20260807 15:15:21.884802 11646 ts_tablet_manager.cc:1918] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:21.884879 11646 log.cc:1196] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/f5e760980f274f5db9eb3d92a018cfab
I20260807 15:15:21.885007 11647 ts_tablet_manager.cc:1939] T f5e760980f274f5db9eb3d92a018cfab P 93c8544156d34bcf88e1e3345f34658f: Deleting consensus metadata
I20260807 15:15:21.885187 11646 ts_tablet_manager.cc:1939] T f5e760980f274f5db9eb3d92a018cfab P 512b8289b81f499ea3023798555c1af0: Deleting consensus metadata
I20260807 15:15:21.885668 10883 catalog_manager.cc:4841] TS 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233): tablet f5e760980f274f5db9eb3d92a018cfab (table test-table [id=4b8179d84e2a4b4caf3e4a298d8f52b8]) successfully deleted
I20260807 15:15:21.886026 10885 catalog_manager.cc:4841] TS 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337): tablet f5e760980f274f5db9eb3d92a018cfab (table test-table [id=4b8179d84e2a4b4caf3e4a298d8f52b8]) successfully deleted
I20260807 15:15:21.886391 11643 ts_tablet_manager.cc:1905] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:15:21.886700 10883 catalog_manager.cc:4808] TS 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233): delete failed for tablet f5e760980f274f5db9eb3d92a018cfab because the tablet was not found. No further retry: Not found: Tablet not found: f5e760980f274f5db9eb3d92a018cfab
I20260807 15:15:21.887318 11088 tablet_service.cc:1467] Processing CreateTablet for tablet c399d8454e0348ee9711f37c10e2dc51 (DEFAULT_TABLE table=test-table [id=3bff5169daac43169cb7f583b58ede92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.887317 11289 tablet_service.cc:1467] Processing CreateTablet for tablet c399d8454e0348ee9711f37c10e2dc51 (DEFAULT_TABLE table=test-table [id=3bff5169daac43169cb7f583b58ede92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.887463 11482 tablet_service.cc:1467] Processing CreateTablet for tablet c399d8454e0348ee9711f37c10e2dc51 (DEFAULT_TABLE table=test-table [id=3bff5169daac43169cb7f583b58ede92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:21.887576 11088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c399d8454e0348ee9711f37c10e2dc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.887574 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c399d8454e0348ee9711f37c10e2dc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.887804 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c399d8454e0348ee9711f37c10e2dc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:21.888872 11643 ts_tablet_manager.cc:1918] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:21.888943 11643 log.cc:1196] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/f5e760980f274f5db9eb3d92a018cfab
I20260807 15:15:21.889196 11643 ts_tablet_manager.cc:1939] T f5e760980f274f5db9eb3d92a018cfab P 2ca88fb8f990457a89f24ee2442b1412: Deleting consensus metadata
I20260807 15:15:21.889923 11610 tablet_bootstrap.cc:492] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Bootstrap starting.
I20260807 15:15:21.889925 11611 tablet_bootstrap.cc:492] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: Bootstrap starting.
I20260807 15:15:21.890172 11612 tablet_bootstrap.cc:492] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Bootstrap starting.
I20260807 15:15:21.890362 10889 catalog_manager.cc:4841] TS 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621): tablet f5e760980f274f5db9eb3d92a018cfab (table test-table [id=4b8179d84e2a4b4caf3e4a298d8f52b8]) successfully deleted
I20260807 15:15:21.890851 11610 tablet_bootstrap.cc:654] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.890851 11611 tablet_bootstrap.cc:654] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.891006 11612 tablet_bootstrap.cc:654] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:21.892078 11610 tablet_bootstrap.cc:492] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: No bootstrap required, opened a new log
I20260807 15:15:21.892171 11610 ts_tablet_manager.cc:1397] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:21.892535 11611 tablet_bootstrap.cc:492] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: No bootstrap required, opened a new log
I20260807 15:15:21.892612 11611 ts_tablet_manager.cc:1397] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:21.892602 11610 raft_consensus.cc:348] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.892731 11610 raft_consensus.cc:374] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.892772 11610 raft_consensus.cc:729] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.892782 11612 tablet_bootstrap.cc:492] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: No bootstrap required, opened a new log
I20260807 15:15:21.892850 11612 ts_tablet_manager.cc:1397] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:21.892901 11610 consensus_queue.cc:260] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.893040 11611 raft_consensus.cc:348] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.893178 11611 raft_consensus.cc:374] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.893198 11610 ts_tablet_manager.cc:1428] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:21.893221 11611 raft_consensus.cc:729] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c8544156d34bcf88e1e3345f34658f, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.893215 11612 raft_consensus.cc:348] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.893318 11612 raft_consensus.cc:374] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:21.893350 11612 raft_consensus.cc:729] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca88fb8f990457a89f24ee2442b1412, State: Initialized, Role: FOLLOWER
I20260807 15:15:21.893342 11611 consensus_queue.cc:260] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.893463 11612 consensus_queue.cc:260] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.893656 11611 ts_tablet_manager.cc:1428] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:21.893760 11612 ts_tablet_manager.cc:1428] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:21.989567 11624 raft_consensus.cc:482] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:21.989748 11624 raft_consensus.cc:504] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.990170 11624 leader_election.cc:288] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.990780 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c399d8454e0348ee9711f37c10e2dc51" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412" is_pre_election: true
I20260807 15:15:21.990779 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c399d8454e0348ee9711f37c10e2dc51" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f" is_pre_election: true
I20260807 15:15:21.990955 11111 raft_consensus.cc:2454] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.990955 11519 raft_consensus.cc:2454] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 0.
I20260807 15:15:21.991310 11223 leader_election.cc:302] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca88fb8f990457a89f24ee2442b1412, 512b8289b81f499ea3023798555c1af0; no voters:
I20260807 15:15:21.991539 11624 raft_consensus.cc:2781] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:21.991607 11624 raft_consensus.cc:482] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:21.991657 11624 raft_consensus.cc:3037] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.992903 11624 raft_consensus.cc:504] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.993314 11624 leader_election.cc:288] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Requested vote from peers 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233), 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621)
I20260807 15:15:21.993726 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c399d8454e0348ee9711f37c10e2dc51" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c8544156d34bcf88e1e3345f34658f"
I20260807 15:15:21.993777 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c399d8454e0348ee9711f37c10e2dc51" candidate_uuid: "512b8289b81f499ea3023798555c1af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca88fb8f990457a89f24ee2442b1412"
I20260807 15:15:21.993873 11111 raft_consensus.cc:3037] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.993911 11519 raft_consensus.cc:3037] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:21.994835 11111 raft_consensus.cc:2454] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.994841 11519 raft_consensus.cc:2454] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b8289b81f499ea3023798555c1af0 in term 1.
I20260807 15:15:21.995172 11219 leader_election.cc:302] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 512b8289b81f499ea3023798555c1af0, 93c8544156d34bcf88e1e3345f34658f; no voters:
I20260807 15:15:21.995374 11624 raft_consensus.cc:2781] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:21.995469 11624 raft_consensus.cc:686] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Becoming Leader. State: Replica: 512b8289b81f499ea3023798555c1af0, State: Running, Role: LEADER
I20260807 15:15:21.995608 11624 consensus_queue.cc:237] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } }
I20260807 15:15:21.997359 10906 catalog_manager.cc:5495] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512b8289b81f499ea3023798555c1af0 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "512b8289b81f499ea3023798555c1af0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "512b8289b81f499ea3023798555c1af0" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:22.034195 11519 raft_consensus.cc:1261] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:22.034245 11111 raft_consensus.cc:1261] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Refusing update from remote peer 512b8289b81f499ea3023798555c1af0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:22.034647 11624 consensus_queue.cc:1035] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca88fb8f990457a89f24ee2442b1412" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:22.034816 11618 consensus_queue.cc:1035] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c8544156d34bcf88e1e3345f34658f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:22.070441 10906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:22.070634 10906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:22.073024 10906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56f1db0a1c2444dc9a8f48e9095d9b7a: Sending DeleteTablet for 3 replicas of tablet c399d8454e0348ee9711f37c10e2dc51
I20260807 15:15:22.073706 11088 tablet_service.cc:1514] Processing DeleteTablet for tablet c399d8454e0348ee9711f37c10e2dc51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:22 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58618
I20260807 15:15:22.073738 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet c399d8454e0348ee9711f37c10e2dc51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:22 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34174
I20260807 15:15:22.073757 11482 tablet_service.cc:1514] Processing DeleteTablet for tablet c399d8454e0348ee9711f37c10e2dc51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:22 UTC) from {username='oryx', principal='oryx/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58940
I20260807 15:15:22.073912 11646 tablet_replica.cc:330] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: stopping tablet replica
I20260807 15:15:22.073912 11647 tablet_replica.cc:330] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: stopping tablet replica
I20260807 15:15:22.073949 11643 tablet_replica.cc:330] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: stopping tablet replica
I20260807 15:15:22.074025 11646 raft_consensus.cc:2229] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:22.074025 11647 raft_consensus.cc:2229] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:22.074061 11643 raft_consensus.cc:2229] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:22.074137 11647 raft_consensus.cc:2258] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:22.074184 11643 raft_consensus.cc:2258] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:22.074223 11646 raft_consensus.cc:2258] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:22.075058 11643 ts_tablet_manager.cc:1905] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:22.075207 11646 ts_tablet_manager.cc:1905] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:22.075335 11647 ts_tablet_manager.cc:1905] T c399d8454e0348ee9711f37c10e2dc51 P 93c8544156d34bcf88e1e3345f34658f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:22.075663 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 11002
I20260807 15:15:22.078120 11646 ts_tablet_manager.cc:1918] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:15:22.078240 11646 log.cc:1196] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/c399d8454e0348ee9711f37c10e2dc51
I20260807 15:15:22.078505 11646 ts_tablet_manager.cc:1939] T c399d8454e0348ee9711f37c10e2dc51 P 512b8289b81f499ea3023798555c1af0: Deleting consensus metadata
I20260807 15:15:22.082062 10885 catalog_manager.cc:4841] TS 512b8289b81f499ea3023798555c1af0 (127.31.230.194:33337): tablet c399d8454e0348ee9711f37c10e2dc51 (table test-table [id=3bff5169daac43169cb7f583b58ede92]) successfully deleted
I20260807 15:15:22.084164 11643 ts_tablet_manager.cc:1918] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:15:22.084259 11643 log.cc:1196] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/c399d8454e0348ee9711f37c10e2dc51
I20260807 15:15:22.084509 11643 ts_tablet_manager.cc:1939] T c399d8454e0348ee9711f37c10e2dc51 P 2ca88fb8f990457a89f24ee2442b1412: Deleting consensus metadata
I20260807 15:15:22.085337 10889 catalog_manager.cc:4841] TS 2ca88fb8f990457a89f24ee2442b1412 (127.31.230.195:36621): tablet c399d8454e0348ee9711f37c10e2dc51 (table test-table [id=3bff5169daac43169cb7f583b58ede92]) successfully deleted
W20260807 15:15:22.091627 10883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:36233 (error 108)
I20260807 15:15:22.091970 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 11191
I20260807 15:15:22.095197 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:22.094786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36233 (local address 127.0.0.1:58624)
0807 15:15:22.095021 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:22.095105 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
W20260807 15:15:22.095470 10883 catalog_manager.cc:4584] TS 93c8544156d34bcf88e1e3345f34658f (127.31.230.193:36233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c399d8454e0348ee9711f37c10e2dc51: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36233: connect: Connection refused (error 111)
I20260807 15:15:22.104096 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 11399
I20260807 15:15:22.118350 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 10860
2026-08-07T15:15:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3412 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:22 dist-test-slave-gkw7 krb5kdc[11738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:22 dist-test-slave-gkw7 krb5kdc[11738](info): set up 2 sockets
Aug 07 15:15:22 dist-test-slave-gkw7 krb5kdc[11738](info): commencing operation
krb5kdc: starting...
W20260807 15:15:23.381457 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:23 dist-test-slave-gkw7 krb5kdc[11738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:23.520857 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38149
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:38149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:23.636358 11756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:23.636688 11756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:23.636778 11756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:23.641803 11756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:23.641861 11756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:23.641881 11756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:23.641897 11756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:23.641937 11756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:23.647806 11756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:38149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:23.648940 11756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:23.650048 11756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:23.658902 11766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:23.658911 11763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:23.658902 11762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:23.659448 11756 server_base.cc:1034] running on GCE node
I20260807 15:15:23.660003 11756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:23.661150 11756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:23.662343 11756 hybrid_clock.cc:648] HybridClock initialized: now 1786115723662319 us; error 48 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T15:15:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1557 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:23 dist-test-slave-gkw7 krb5kdc[11774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:23 dist-test-slave-gkw7 krb5kdc[11774](info): set up 2 sockets
Aug 07 15:15:23 dist-test-slave-gkw7 krb5kdc[11774](info): commencing operation
krb5kdc: starting...
W20260807 15:15:24.949606 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:25.086226 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38555
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:38555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:25.202159 11789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:25.202493 11789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:25.202576 11789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:25.206645 11789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:25.206738 11789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:25.206776 11789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:25.206833 11789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:25.206872 11789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:25.212271 11789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:38555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:25.213402 11789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:25.214486 11789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:25.223631 11797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.223676 11803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.223625 11799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:25.223918 11789 server_base.cc:1034] running on GCE node
I20260807 15:15:25.224658 11789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:25.225993 11789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:25.227206 11789 hybrid_clock.cc:648] HybridClock initialized: now 1786115725227180 us; error 40 us; skew 500 ppm
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:25.230444 11789 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:25.231356 11789 webserver.cc:460] Webserver started at http://127.31.230.254:33263/ using document root <none> and password file <none>
I20260807 15:15:25.231959 11789 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:25.232052 11789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:25.232303 11789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:25.234493 11789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "c4b7426440354d8f8b5c544832366030"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "e8f7c15276a2c0ff0185ed5039780abb"
server_key_iv: "95e7973c8d4f326293e3ce8f9c2f4f9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.235092 11789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "c4b7426440354d8f8b5c544832366030"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "e8f7c15276a2c0ff0185ed5039780abb"
server_key_iv: "95e7973c8d4f326293e3ce8f9c2f4f9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.238919 11789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:25.241585 11815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.242935 11789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:15:25.243181 11789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "c4b7426440354d8f8b5c544832366030"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "e8f7c15276a2c0ff0185ed5039780abb"
server_key_iv: "95e7973c8d4f326293e3ce8f9c2f4f9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.243301 11789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:25.260231 11789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:25.263675 11789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:25.263936 11789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:25.271500 11789 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:38555
I20260807 15:15:25.271525 11889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:38555 every 8 connection(s)
I20260807 15:15:25.272677 11789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:25.274650 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 11789
I20260807 15:15:25.274794 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:25.275156 32667 external_mini_cluster.cc:1428] Setting key c2ddeb785c88ead52bafc77a13522091
I20260807 15:15:25.277046 11890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:25.285043 11890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Bootstrap starting.
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:25.287845 11890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:25.288617 11890 log.cc:824] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:25.290853 11890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: No bootstrap required, opened a new log
I20260807 15:15:25.291683 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.276956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.31.230.254:38555)
0807 15:15:25.277468 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:25.277488 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:25.277581 (+ 93us) server_negotiation.cc:407] Connection header received
0807 15:15:25.278282 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:25.278299 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:25.278651 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:25.279027 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:25.280013 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.281760 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.283121 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.283631 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.283762 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:25.286703 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:25.286710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:25.286726 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:25.286755 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:25.288598 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:25.289104 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:25.289109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:25.289114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:25.289244 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:25.289575 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:25.289582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:25.289586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:25.290012 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:25.290238 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:25.290537 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 15:15:25.290855 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":184,"threads_started":1}
I20260807 15:15:25.295035 11890 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:25.295301 11890 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:25.295388 11890 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b7426440354d8f8b5c544832366030, State: Initialized, Role: FOLLOWER
I20260807 15:15:25.295845 11890 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:25.296007 11890 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:25.296096 11890 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:25.296224 11890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:25.297214 11890 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:25.297659 11890 leader_election.cc:302] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4b7426440354d8f8b5c544832366030; no voters:
I20260807 15:15:25.298049 11890 leader_election.cc:288] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:25.298210 11898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:25.298499 11898 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 LEADER]: Becoming Leader. State: Replica: c4b7426440354d8f8b5c544832366030, State: Running, Role: LEADER
I20260807 15:15:25.298918 11898 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:25.299480 11890 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:25.300908 11899 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4b7426440354d8f8b5c544832366030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } } }
I20260807 15:15:25.300949 11900 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4b7426440354d8f8b5c544832366030. Latest consensus state: current_term: 1 leader_uuid: "c4b7426440354d8f8b5c544832366030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } } }
I20260807 15:15:25.301141 11899 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:25.301177 11900 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: This master's current role is: LEADER
W20260807 15:15:25.304988 11919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:25.305081 11919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:25.305140 11915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:25.306087 11915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:25.312052 11915 catalog_manager.cc:1347] Generated new cluster ID: 8af3a92f74444e77b58ff2af43362330
I20260807 15:15:25.312144 11915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:25.321476 11915 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:25.322850 11915 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:25.337520 11915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Generated new TSK 0
I20260807 15:15:25.338348 11915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:15:25.380920 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:38555
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:25.503417 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:25.503734 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:25.503785 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:25.508318 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:25.508394 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:25.508459 11928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:25.514238 11928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:38555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:25.515395 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:25.516608 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:25.527154 11940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.527174 11937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.527184 11938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:25.527510 11928 server_base.cc:1034] running on GCE node
I20260807 15:15:25.527963 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:25.528734 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:25.529930 11928 hybrid_clock.cc:648] HybridClock initialized: now 1786115725529914 us; error 33 us; skew 500 ppm
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:25.532897 11928 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:15:25.533931 11928 webserver.cc:460] Webserver started at http://127.31.230.193:44827/ using document root <none> and password file <none>
I20260807 15:15:25.534497 11928 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:25.534559 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:25.534739 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:25.536798 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "bfc3372a4e974f8cb0e9bef7771f50f5"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "36e00dfc2199a30c0b2559772f3ab6b1"
server_key_iv: "de90e88dabea88dcba36e708770405b7"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.537325 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "bfc3372a4e974f8cb0e9bef7771f50f5"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "36e00dfc2199a30c0b2559772f3ab6b1"
server_key_iv: "de90e88dabea88dcba36e708770405b7"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.541244 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:25.544104 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.545737 11928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:25.545866 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "bfc3372a4e974f8cb0e9bef7771f50f5"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "36e00dfc2199a30c0b2559772f3ab6b1"
server_key_iv: "de90e88dabea88dcba36e708770405b7"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.546021 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:25.556347 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:25.559625 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:25.559846 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:25.560456 11928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:25.561331 11928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:25.561403 11928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.561475 11928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:25.561527 11928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.570590 11928 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:33053
I20260807 15:15:25.570662 12114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:33053 every 8 connection(s)
I20260807 15:15:25.571563 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:25.578800 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 11928
I20260807 15:15:25.578935 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:25.579299 32667 external_mini_cluster.cc:1428] Setting key 1cca27d60bb38926210f735d05109c9b
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:25.585988 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.573484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:56665 (local address 127.31.230.254:38555)
0807 15:15:25.573639 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:25.573643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:25.574744 (+ 1101us) server_negotiation.cc:407] Connection header received
0807 15:15:25.575570 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:25.575572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:25.575621 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:25.575696 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:25.576907 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.577445 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.578692 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.579079 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.579195 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:25.581979 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:25.581985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:25.581991 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:25.582026 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:25.583651 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:25.584327 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:25.584330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:25.584331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:25.584410 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:25.584842 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:25.584844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:25.584845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:25.585015 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:25.585088 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:25.585755 (+ 667us) server_negotiation.cc:299] Negotiation successful
0807 15:15:25.585866 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:15:25.587354 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.573860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:56665)
0807 15:15:25.574571 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:25.574618 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:25.575390 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:25.575871 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:25.575878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:25.576284 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:25.576770 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:25.576784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.577631 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.577634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:25.578564 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:25.578573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.579231 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.579234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:25.579375 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:25.580047 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:25.580063 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:25.581811 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:25.583786 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:25.583794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:25.583813 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:25.584212 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:25.584504 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:25.584509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:25.584512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:25.584737 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:25.585121 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:25.585129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:25.585452 (+ 323us) client_negotiation.cc:769] Sending connection context
0807 15:15:25.585693 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 15:15:25.586069 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":117,"threads_started":1}
I20260807 15:15:25.588769 12115 heartbeater.cc:347] Connected to a master server at 127.31.230.254:38555
I20260807 15:15:25.589026 12115 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:25.589581 12115 heartbeater.cc:511] Master 127.31.230.254:38555 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:15:25.591240 11843 ts_manager.cc:194] Registered new tserver with Master: bfc3372a4e974f8cb0e9bef7771f50f5 (127.31.230.193:33053)
I20260807 15:15:25.592442 11843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:56665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:15:25.613078 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:38555
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:25.730104 12122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:25.730381 12122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:25.730463 12122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:25.734452 12122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:25.734510 12122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:25.734570 12122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:25.739970 12122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:38555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:25.741117 12122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:25.742228 12122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:25.752568 12130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.752571 12137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.752571 12132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:25.752839 12122 server_base.cc:1034] running on GCE node
I20260807 15:15:25.753372 12122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:25.754180 12122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:25.755358 12122 hybrid_clock.cc:648] HybridClock initialized: now 1786115725755341 us; error 27 us; skew 500 ppm
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:25.758325 12122 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:15:25.759374 12122 webserver.cc:460] Webserver started at http://127.31.230.194:40535/ using document root <none> and password file <none>
I20260807 15:15:25.760005 12122 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:25.760115 12122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:25.760378 12122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:25.762843 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "de0156854b4d441f9c80718c2ae9fc75"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "af42a06492a573803b074903a2b4c09b"
server_key_iv: "79e68112847c66207bec563c3149a5fe"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.763463 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "de0156854b4d441f9c80718c2ae9fc75"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "af42a06492a573803b074903a2b4c09b"
server_key_iv: "79e68112847c66207bec563c3149a5fe"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.767611 12122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:25.770316 12151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.771612 12122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:25.771763 12122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "de0156854b4d441f9c80718c2ae9fc75"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "af42a06492a573803b074903a2b4c09b"
server_key_iv: "79e68112847c66207bec563c3149a5fe"
server_key_version: "encryptionkey@0"
I20260807 15:15:25.771883 12122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:25.798481 12122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:25.802141 12122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:25.802378 12122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:25.803059 12122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:25.804001 12122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:25.804087 12122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.804212 12122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:25.804261 12122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:25.814358 12122 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:37043
I20260807 15:15:25.814388 12312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:37043 every 8 connection(s)
I20260807 15:15:25.815383 12122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:25.821053 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 12122
I20260807 15:15:25.821197 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:25.821489 32667 external_mini_cluster.cc:1428] Setting key 85688a4eb88f59aa112d6329889eeab1
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:25.830385 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.817386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:33901 (local address 127.31.230.254:38555)
0807 15:15:25.817577 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:25.817581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:25.818707 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 15:15:25.819658 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:25.819661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:25.819730 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:25.819841 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:25.821230 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.821943 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.823344 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.823747 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.823873 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:25.826466 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:25.826470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:25.826472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:25.826498 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:25.828113 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:25.828697 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:25.828703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:25.828705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:25.828795 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:25.829156 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:25.829161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:25.829162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:25.829369 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:25.829484 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:25.830066 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 15:15:25.830203 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:15:25.831280 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.817870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:33901)
0807 15:15:25.818550 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:25.818593 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:25.819428 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:25.820021 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:25.820028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:25.820392 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:25.820969 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:25.820992 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.822082 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.822088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:25.823218 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:25.823228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:25.823901 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:25.823904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:25.824049 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:25.824684 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:25.824700 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:25.826280 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:25.828284 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:25.828291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:25.828308 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:25.828583 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:25.828897 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:25.828900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:25.828902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:25.829054 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:25.829485 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:25.829492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:25.829813 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 15:15:25.830022 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 15:15:25.830376 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:15:25.832621 12313 heartbeater.cc:347] Connected to a master server at 127.31.230.254:38555
I20260807 15:15:25.832921 12313 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:25.833564 12313 heartbeater.cc:511] Master 127.31.230.254:38555 requested a full tablet report, sending...
I20260807 15:15:25.834724 11843 ts_manager.cc:194] Registered new tserver with Master: de0156854b4d441f9c80718c2ae9fc75 (127.31.230.194:37043)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:15:25.835539 11843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:33901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:15:25.853005 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:38555
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:25.968333 12318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:25.968665 12318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:25.968757 12318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:25.972790 12318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:25.972895 12318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:25.972994 12318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:25.979363 12318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:38555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:25.980535 12318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:25.981609 12318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:25.991755 12325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:25.991879 12318 server_base.cc:1034] running on GCE node
W20260807 15:15:25.991777 12323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:25.992074 12327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:25.992553 12318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:25.993255 12318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:25.994453 12318 hybrid_clock.cc:648] HybridClock initialized: now 1786115725994411 us; error 58 us; skew 500 ppm
Aug 07 15:15:25 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:25.997511 12318 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:15:25.998505 12318 webserver.cc:460] Webserver started at http://127.31.230.195:41339/ using document root <none> and password file <none>
I20260807 15:15:25.999204 12318 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:25.999302 12318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:25.999552 12318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:26.002279 12318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "0e3f9b024aea4cfd88f9743a998f3dff"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "de65c5aad7ec8c09ee69d75b415e97f7"
server_key_iv: "ac4fe7e2179fb4c5b652fe4c9b34742e"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.002954 12318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "0e3f9b024aea4cfd88f9743a998f3dff"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "de65c5aad7ec8c09ee69d75b415e97f7"
server_key_iv: "ac4fe7e2179fb4c5b652fe4c9b34742e"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.006948 12318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:26.009758 12339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.011070 12318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:15:26.011229 12318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0e3f9b024aea4cfd88f9743a998f3dff"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "de65c5aad7ec8c09ee69d75b415e97f7"
server_key_iv: "ac4fe7e2179fb4c5b652fe4c9b34742e"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.011358 12318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:26.033015 12318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:26.036304 12318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:26.036543 12318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:26.037209 12318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:26.038189 12318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:26.038271 12318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.038347 12318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:26.038390 12318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.048290 12318 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:40505
I20260807 15:15:26.048319 12494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:40505 every 8 connection(s)
I20260807 15:15:26.049273 12318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:26.050559 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 12318
I20260807 15:15:26.050721 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:26.051056 32667 external_mini_cluster.cc:1428] Setting key f44fef80fdc6a623c443fd716b74bddd
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115725, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:26.063476 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.051671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:49195 (local address 127.31.230.254:38555)
0807 15:15:26.051836 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.051841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.053035 (+ 1194us) server_negotiation.cc:407] Connection header received
0807 15:15:26.054140 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.054143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.054189 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.054271 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.055456 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.055987 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.057165 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.057401 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.057494 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.060133 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:26.060136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:26.060138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:26.060161 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:26.061583 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:26.062081 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:26.062084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:26.062085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:26.062166 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:26.062487 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:26.062490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:26.062491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:26.062670 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:26.062745 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:26.063224 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 15:15:26.063347 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:15:26.064589 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.051999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:49195)
0807 15:15:26.052860 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.052925 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.053942 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.054413 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.054420 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.054801 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.055315 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.055330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.056099 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.056102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.057044 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.057054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.057598 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.057603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.057768 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.058432 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.058450 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.059975 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:26.061698 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:26.061703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:26.061720 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:26.061991 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:26.062262 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:26.062264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:26.062265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:26.062405 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:26.062761 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:26.062765 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:26.063003 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 15:15:26.063173 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 15:15:26.063465 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":157,"threads_started":1}
I20260807 15:15:26.065903 12496 heartbeater.cc:347] Connected to a master server at 127.31.230.254:38555
I20260807 15:15:26.066148 12496 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:26.066687 12496 heartbeater.cc:511] Master 127.31.230.254:38555 requested a full tablet report, sending...
I20260807 15:15:26.067757 11843 ts_manager.cc:194] Registered new tserver with Master: 0e3f9b024aea4cfd88f9743a998f3dff (127.31.230.195:40505)
I20260807 15:15:26.068342 11843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:49195
I20260807 15:15:26.077932 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab.
Entry for principal oryx/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab.
WARNING: no policy specified for oryx/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab.
Entry for principal oryx/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab.
WARNING: no policy specified for oryx/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab.
Entry for principal oryx/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab.
I20260807 15:15:26.161518 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 11928
I20260807 15:15:26.181185 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 12122
I20260807 15:15:26.195166 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 12318
I20260807 15:15:26.207544 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 11789
I20260807 15:15:26.217958 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38555
--webserver_interface=127.31.230.254
--webserver_port=33263
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:38555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:26.333133 12510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:26.333423 12510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:26.333465 12510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:26.337517 12510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:26.337607 12510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:26.337627 12510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:26.337644 12510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:26.337659 12510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:26.342955 12510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:38555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=33263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:26.344017 12510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:26.345131 12510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:26.354070 12522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.354056 12517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.354074 12519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:26.354167 12510 server_base.cc:1034] running on GCE node
I20260807 15:15:26.354880 12510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:26.356101 12510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:26.357298 12510 hybrid_clock.cc:648] HybridClock initialized: now 1786115726357265 us; error 46 us; skew 500 ppm
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115726, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:26.360334 12510 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:26.361299 12510 webserver.cc:460] Webserver started at http://127.31.230.254:33263/ using document root <none> and password file <none>
I20260807 15:15:26.361940 12510 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:26.362025 12510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:26.366041 12510 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:15:26.368197 12532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.369478 12510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:26.369691 12510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "c4b7426440354d8f8b5c544832366030"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "e8f7c15276a2c0ff0185ed5039780abb"
server_key_iv: "95e7973c8d4f326293e3ce8f9c2f4f9f"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.370250 12510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:26.383260 12510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:26.386812 12510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:26.387037 12510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:26.394742 12510 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:38555
I20260807 15:15:26.394773 12600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:38555 every 8 connection(s)
I20260807 15:15:26.396044 12510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:26.400665 12602 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:15:26.402359 12602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Bootstrap starting.
I20260807 15:15:26.405269 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 12510
I20260807 15:15:26.405818 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:33053
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:38555
--webserver_port=44827
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:26.420549 12602 log.cc:824] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:26.427800 12602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:15:26.428227 12602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Bootstrap complete.
I20260807 15:15:26.432739 12602 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:26.433192 12602 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b7426440354d8f8b5c544832366030, State: Initialized, Role: FOLLOWER
I20260807 15:15:26.433812 12602 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:26.433944 12602 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:26.434032 12602 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:26.434130 12602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:15:26.436305 12602 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:26.436671 12602 leader_election.cc:302] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4b7426440354d8f8b5c544832366030; no voters:
I20260807 15:15:26.437064 12602 leader_election.cc:288] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:15:26.437160 12608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:15:26.437436 12608 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [term 2 LEADER]: Becoming Leader. State: Replica: c4b7426440354d8f8b5c544832366030, State: Running, Role: LEADER
I20260807 15:15:26.437829 12608 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } }
I20260807 15:15:26.438437 12602 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:26.439445 12610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4b7426440354d8f8b5c544832366030. Latest consensus state: current_term: 2 leader_uuid: "c4b7426440354d8f8b5c544832366030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } } }
I20260807 15:15:26.439576 12610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:26.439503 12609 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c4b7426440354d8f8b5c544832366030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b7426440354d8f8b5c544832366030" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38555 } } }
I20260807 15:15:26.439625 12609 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:26.444476 12627 catalog_manager.cc:1259] Loaded cluster ID: 8af3a92f74444e77b58ff2af43362330
I20260807 15:15:26.444558 12627 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: loading cluster ID for follower catalog manager: success
I20260807 15:15:26.447024 12627 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: acquiring CA information for follower catalog manager: success
I20260807 15:15:26.448414 12627 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:15:26.448539 12620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:26.449426 12620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:26.449914 12620 catalog_manager.cc:1259] Loaded cluster ID: 8af3a92f74444e77b58ff2af43362330
I20260807 15:15:26.449977 12620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:26.450639 12620 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:26.451125 12620 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c4b7426440354d8f8b5c544832366030: Loaded TSK: 0
I20260807 15:15:26.451917 12620 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:15:26.537176 12605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:26.537467 12605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:26.537511 12605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:26.542140 12605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:26.542217 12605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:26.542300 12605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:26.547788 12605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.193.keytab
--principal=oryx/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:33053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=44827
--tserver_master_addrs=127.31.230.254:38555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:26.549114 12605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:26.550324 12605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:26.560525 12640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.560542 12642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.560525 12644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:26.561101 12605 server_base.cc:1034] running on GCE node
I20260807 15:15:26.561653 12605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:26.562357 12605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:26.563549 12605 hybrid_clock.cc:648] HybridClock initialized: now 1786115726563530 us; error 35 us; skew 500 ppm
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115726, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:26.566514 12605 init.cc:377] Logged in from keytab as oryx/127.31.230.193@KRBTEST.COM (short username oryx)
I20260807 15:15:26.567521 12605 webserver.cc:460] Webserver started at http://127.31.230.193:44827/ using document root <none> and password file <none>
I20260807 15:15:26.568104 12605 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:26.568212 12605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:26.572373 12605 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:26.574879 12654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.576200 12605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:15:26.576324 12605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "bfc3372a4e974f8cb0e9bef7771f50f5"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "36e00dfc2199a30c0b2559772f3ab6b1"
server_key_iv: "de90e88dabea88dcba36e708770405b7"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.576862 12605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:26.592820 12605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:26.596357 12605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:26.596531 12605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:26.597199 12605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:26.598110 12605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:26.598160 12605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.598206 12605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:26.598258 12605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.608038 12605 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:33053
I20260807 15:15:26.608067 12815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:33053 every 8 connection(s)
I20260807 15:15:26.609057 12605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:26.614368 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 12605
I20260807 15:15:26.614923 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:37043
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:38555
--webserver_port=40535
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.623731 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.611243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:54429)
0807 15:15:26.611969 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.612012 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.612798 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.614189 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.614200 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.614736 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.615461 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.615485 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.616994 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.616997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.617980 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.617990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.618836 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.618843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.619077 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.619793 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.619839 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.622699 (+ 2860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":162,"threads_started":1}
W20260807 15:15:26.623731 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:26.611172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:54429 (local address 127.31.230.254:38555)
0807 15:15:26.611676 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.611697 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.612141 (+ 444us) server_negotiation.cc:407] Connection header received
0807 15:15:26.613225 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.613237 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.613715 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.614028 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.615666 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.616830 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.618120 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.618643 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.618795 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.622943 (+ 4148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:54429: BlockingRecv error: recv got EOF from 127.31.230.193:54429 (error 108)
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":102,"threads_started":1}
W20260807 15:15:26.624250 12816 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:38555 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.230.254:38555: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.630091 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.624779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:60979)
0807 15:15:26.625002 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.625019 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.625121 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.625389 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.625392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.625614 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.625718 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.625724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.626452 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.626457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.627021 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.627030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.627488 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.627492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.627584 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.628098 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.628111 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.629947 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260807 15:15:26.630246 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:26.624866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:60979 (local address 127.31.230.254:38555)
0807 15:15:26.625015 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.625027 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.625069 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:15:26.625220 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.625226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.625284 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.625370 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.625832 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.626340 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.627155 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.627378 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.627484 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.630118 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:60979: BlockingRecv error: recv got EOF from 127.31.230.193:60979 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.635735 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.630776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:33219)
0807 15:15:26.630894 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.630909 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.630988 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.631239 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.631242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.631394 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.631467 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.631472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.632192 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.632196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.632716 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.632724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.633178 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.633182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.633278 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.633843 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.633854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.635602 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 15:15:26.635872 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:26.630849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:33219 (local address 127.31.230.254:38555)
0807 15:15:26.630972 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.630976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.630990 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:26.631082 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.631085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.631133 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.631215 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.631582 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.632080 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.632846 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.633067 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.633175 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.635735 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:33219: BlockingRecv error: recv got EOF from 127.31.230.193:33219 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 15:15:26.635968 12816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:15:26.734993 12823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:26.735270 12823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:26.735359 12823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:26.739598 12823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:26.739665 12823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:26.739797 12823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:26.745200 12823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.194.keytab
--principal=oryx/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:37043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=40535
--tserver_master_addrs=127.31.230.254:38555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:26.746371 12823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:26.747427 12823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:26.757431 12832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.757405 12830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.757405 12834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:26.758105 12823 server_base.cc:1034] running on GCE node
I20260807 15:15:26.758534 12823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:26.759114 12823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:26.760280 12823 hybrid_clock.cc:648] HybridClock initialized: now 1786115726760248 us; error 45 us; skew 500 ppm
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115726, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:26.763165 12823 init.cc:377] Logged in from keytab as oryx/127.31.230.194@KRBTEST.COM (short username oryx)
I20260807 15:15:26.764047 12823 webserver.cc:460] Webserver started at http://127.31.230.194:40535/ using document root <none> and password file <none>
I20260807 15:15:26.764626 12823 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:26.764727 12823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:26.768769 12823 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:26.770924 12842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.772171 12823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:26.772320 12823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "de0156854b4d441f9c80718c2ae9fc75"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "af42a06492a573803b074903a2b4c09b"
server_key_iv: "79e68112847c66207bec563c3149a5fe"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.772861 12823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:26.787825 12823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:26.790866 12823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:26.791085 12823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:26.791718 12823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:26.792577 12823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:26.792625 12823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.792699 12823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:26.792743 12823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.802042 12823 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:37043
I20260807 15:15:26.802094 12997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:37043 every 8 connection(s)
I20260807 15:15:26.803504 12823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.813346 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 12823
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.813928 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:40505
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:38555
--webserver_port=41339
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:26.815152 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:26.805466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:53211 (local address 127.31.230.254:38555)
0807 15:15:26.805683 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.805687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.806749 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 15:15:26.807705 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.807718 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.807795 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.807891 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.809086 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.809689 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.810850 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.811081 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.811192 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.814972 (+ 3780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:53211: BlockingRecv error: recv got EOF from 127.31.230.194:53211 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
I20260807 15:15:26.815646 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.805910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:53211)
0807 15:15:26.806588 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.806630 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.807479 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.808077 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.808084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.808451 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.808935 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.808947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.809829 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.809832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.810716 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.810726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.811256 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.811262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.811423 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.812050 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.812067 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.814786 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":106,"threads_started":1}
W20260807 15:15:26.816148 12998 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:38555 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.230.254:38555: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.822284 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.816656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:35283)
0807 15:15:26.816783 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.816799 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.816885 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.817157 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.817160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.817324 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.817414 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.817419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.818502 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.818504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.819047 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.819055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.819727 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.819729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.819799 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.820307 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.820319 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.822171 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:15:26.822820 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:26.816746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:35283 (local address 127.31.230.254:38555)
0807 15:15:26.816866 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.816871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.816884 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:26.816995 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.816998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.817051 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.817139 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.817574 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.818380 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.819176 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.819607 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.819741 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.822387 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:35283: BlockingRecv error: recv got EOF from 127.31.230.194:35283 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:26.828436 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.822886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:59281)
0807 15:15:26.823063 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.823080 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:26.823168 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:26.823472 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:26.823474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:26.823600 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:26.823672 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.823677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.824757 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.824759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.825272 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:26.825279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.825949 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.825951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:26.826011 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.826517 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:26.826528 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:26.828305 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260807 15:15:26.828660 12998 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:15:26.828715 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:26.822959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59281 (local address 127.31.230.254:38555)
0807 15:15:26.823092 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:26.823100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:26.823122 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:15:26.823286 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:26.823291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:26.823350 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:26.823447 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:26.823793 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.824643 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.825394 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:26.825833 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:26.825968 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:26.828433 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:59281: BlockingRecv error: recv got EOF from 127.31.230.194:59281 (error 108)
Metrics: {"server-negotiator.queue_time_us":50,"thread_start_us":92,"threads_started":1}
W20260807 15:15:26.932003 13002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:26.932293 13002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:26.932380 13002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:26.936511 13002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:26.936584 13002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:26.936687 13002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:26.942812 13002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/krb5kdc/oryx_127.31.230.195.keytab
--principal=oryx/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:40505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=41339
--tserver_master_addrs=127.31.230.254:38555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:26.943933 13002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:26.945019 13002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:26.955582 13015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:26.955596 13011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:26.955638 13002 server_base.cc:1034] running on GCE node
W20260807 15:15:26.955598 13010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:26.956271 13002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:26.956949 13002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:26.958112 13002 hybrid_clock.cc:648] HybridClock initialized: now 1786115726958093 us; error 28 us; skew 500 ppm
Aug 07 15:15:26 dist-test-slave-gkw7 krb5kdc[11774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115726, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:26.960930 13002 init.cc:377] Logged in from keytab as oryx/127.31.230.195@KRBTEST.COM (short username oryx)
I20260807 15:15:26.961939 13002 webserver.cc:460] Webserver started at http://127.31.230.195:41339/ using document root <none> and password file <none>
I20260807 15:15:26.962532 13002 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:26.962633 13002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:26.966642 13002 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:15:26.968815 13025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.970139 13002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:26.970279 13002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0e3f9b024aea4cfd88f9743a998f3dff"
format_stamp: "Formatted at 2026-08-07 15:15:25 on dist-test-slave-gkw7"
server_key: "de65c5aad7ec8c09ee69d75b415e97f7"
server_key_iv: "ac4fe7e2179fb4c5b652fe4c9b34742e"
server_key_version: "encryptionkey@0"
I20260807 15:15:26.970819 13002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:26.990770 13002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:26.994158 13002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:26.994401 13002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:26.995126 13002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:26.996070 13002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:26.996122 13002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:26.996199 13002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:26.996244 13002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:27.007375 13002 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:40505
I20260807 15:15:27.007400 13179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:40505 every 8 connection(s)
I20260807 15:15:27.008567 13002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:27.012795 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 13002
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:15:27.020699 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:27.010571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:41395 (local address 127.31.230.254:38555)
0807 15:15:27.010743 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:27.010748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:27.011889 (+ 1141us) server_negotiation.cc:407] Connection header received
0807 15:15:27.012769 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:27.012774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:27.012838 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:27.012948 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:27.014620 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.015345 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.016660 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.016918 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.017057 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.020548 (+ 3491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:41395: BlockingRecv error: recv got EOF from 127.31.230.195:41395 (error 108)
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":4864}
I20260807 15:15:27.021351 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:27.011034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:41395)
0807 15:15:27.011723 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:27.011765 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:27.012549 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:27.013145 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:27.013153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:27.013765 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:27.014456 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.014474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.015480 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.015484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.016496 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.016507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.017083 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.017089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.017282 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.017924 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:27.017940 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:27.020381 (+ 2441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":119,"threads_started":1}
W20260807 15:15:27.021966 13181 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:38555 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.230.254:38555: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
I20260807 15:15:27.023177 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:27.013915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.31.230.254:38555)
0807 15:15:27.014063 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:27.014068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:27.014084 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:15:27.014121 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:27.014124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:27.014163 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:27.014258 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:27.015081 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.015737 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.016623 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.016917 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.017027 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.018659 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:27.018665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:27.018679 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:27.018704 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:27.020601 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:27.021127 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:27.021132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:27.021137 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:27.021257 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:27.021615 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:27.021623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:27.021624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:27.022131 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:27.022316 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:27.022815 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 15:15:27.023067 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":60}
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:27.028092 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:27.022458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:34475)
0807 15:15:27.022623 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:27.022638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:27.022714 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:27.022970 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:27.022973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:27.023109 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:27.023200 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.023229 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.024215 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.024218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.025018 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.025027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.025607 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.025611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.025705 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.026196 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:27.026240 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:27.027955 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104}
W20260807 15:15:27.028177 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:27.022544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:34475 (local address 127.31.230.254:38555)
0807 15:15:27.022660 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:27.022664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:27.022677 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:27.022806 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:27.022809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:27.022849 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:27.022927 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:27.023385 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.024092 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.025162 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.025470 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.025605 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.028063 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:34475: BlockingRecv error: recv got EOF from 127.31.230.195:34475 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:27.033084 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:27.028641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:43949)
0807 15:15:27.028755 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:27.028771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:27.028841 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:27.029039 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:27.029041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:27.029161 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:27.029247 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.029252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.029959 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.029961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.030493 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.030501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.030883 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.030885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.030942 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.031336 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:27.031347 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:27.033000 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:15:27.033221 12818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:27.028728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:43949 (local address 127.31.230.254:38555)
0807 15:15:27.028873 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:27.028877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:27.028887 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:27.028915 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:27.028917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:27.028947 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:27.029010 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:27.029343 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.029866 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.030591 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.030787 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.030859 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.033110 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:43949: BlockingRecv error: recv got EOF from 127.31.230.195:43949 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 15:15:27.033378 13181 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:27.643155 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:27.636808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:58319)
0807 15:15:27.637152 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:27.637168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:27.637311 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:27.637659 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:27.637665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:27.637905 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:27.638073 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.638089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.639008 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.639011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.639781 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.639791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.640284 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.640287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.640361 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.640894 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:27.640909 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:27.643014 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":106,"threads_started":1}
W20260807 15:15:27.643430 13188 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:27.636907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:58319 (local address 127.31.230.254:38555)
0807 15:15:27.637209 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:27.637214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:27.637229 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:27.637444 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:27.637449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:27.637513 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:27.637628 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:27.638227 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.638908 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.639894 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.640148 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.640250 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.643160 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:58319: BlockingRecv error: recv got EOF from 127.31.230.193:58319 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:27 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:27.834707 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:27.829356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:44549)
0807 15:15:27.829686 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:27.829700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:27.829820 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:27.830063 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:27.830065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:27.830257 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:27.830380 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.830386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.831091 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.831093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.831676 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:27.831685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.832089 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.832091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:27.832164 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.832675 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:27.832686 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:27.834579 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
W20260807 15:15:27.834853 13188 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:27.829430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:44549 (local address 127.31.230.254:38555)
0807 15:15:27.829635 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:27.829640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:27.829779 (+ 139us) server_negotiation.cc:407] Connection header received
0807 15:15:27.829902 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:27.829904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:27.829953 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:27.830024 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:27.830479 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.830999 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.831776 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:27.831994 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:27.832068 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:27.834729 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:44549: BlockingRecv error: recv got EOF from 127.31.230.194:44549 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 15:15:28 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:28 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:28.039819 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.034070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:52673)
0807 15:15:28.034377 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.034391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.034495 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.034747 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.034749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.034936 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:28.035065 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.035070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.035810 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.035813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.036398 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.036406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.036854 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.036857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.036936 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.037455 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:28.037467 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:28.039685 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":102,"threads_started":1}
W20260807 15:15:28.040037 13188 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:28.034176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:52673 (local address 127.31.230.254:38555)
0807 15:15:28.034339 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.034342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.034463 (+ 121us) server_negotiation.cc:407] Connection header received
0807 15:15:28.034577 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.034580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.034629 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.034701 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.035168 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.035708 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.036514 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.036742 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.036826 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.039902 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:52673: BlockingRecv error: recv got EOF from 127.31.230.195:52673 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 15:15:28 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:28 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:28.649956 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.644013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:51643)
0807 15:15:28.644313 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.644327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.644429 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.644692 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.644694 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.644882 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:28.644987 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.644993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.645821 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.645824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.646413 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.646422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.646962 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.646965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.647070 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.647735 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:28.647755 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:28.649824 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
W20260807 15:15:28.650110 13192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:28.644106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:51643 (local address 127.31.230.254:38555)
0807 15:15:28.644408 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.644412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.644428 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:15:28.644521 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.644524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.644571 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.644655 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.645094 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.645698 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.646523 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.646811 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.646936 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.649970 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:51643: BlockingRecv error: recv got EOF from 127.31.230.193:51643 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
Aug 07 15:15:28 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:28 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:28.846414 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.835616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:49461)
0807 15:15:28.835895 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.835909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.836013 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.836267 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.836269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.836456 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:28.836558 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.836563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.837305 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.837307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.837897 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.837905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.838348 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.838355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.838470 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.843996 (+ 5526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:28.844011 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:28.846274 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
W20260807 15:15:28.846643 13192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:28.835684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:49461 (local address 127.31.230.254:38555)
0807 15:15:28.835827 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.835831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.835995 (+ 164us) server_negotiation.cc:407] Connection header received
0807 15:15:28.836093 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.836095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.836142 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.836212 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.836656 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.837193 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.837995 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.838216 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.838344 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.846402 (+ 8058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:49461: BlockingRecv error: recv got EOF from 127.31.230.194:49461 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 15:15:29 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:29 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:29.046749 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.040757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:47285)
0807 15:15:29.041096 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.041113 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.041235 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.041548 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.041551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.041745 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:29.041864 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.041871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.042610 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.042612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.043160 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.043167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.043857 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.043861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.043969 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.044642 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:29.044655 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:29.046614 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":128,"threads_started":1}
W20260807 15:15:29.046877 13192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:29.040910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:47285 (local address 127.31.230.254:38555)
0807 15:15:29.041043 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.041047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.041188 (+ 141us) server_negotiation.cc:407] Connection header received
0807 15:15:29.041323 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.041325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.041377 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.041456 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.041974 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.042514 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.043341 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.043691 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.043841 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.046757 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:47285: BlockingRecv error: recv got EOF from 127.31.230.195:47285 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 15:15:29 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:29 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:29.656787 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.650921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:47589)
0807 15:15:29.651209 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.651228 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.651352 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.651594 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.651597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.651797 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:29.651901 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.651906 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.652641 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.652643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.653216 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.653224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.653922 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.653925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.654024 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.654616 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:29.654636 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:29.656644 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
W20260807 15:15:29.656909 13198 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:29.651047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:47589 (local address 127.31.230.254:38555)
0807 15:15:29.651368 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.651372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.651387 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:15:29.651430 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.651432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.651477 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.651560 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.652011 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.652547 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.653345 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.653706 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.653831 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.656779 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:47589: BlockingRecv error: recv got EOF from 127.31.230.193:47589 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":117,"threads_started":1}
Aug 07 15:15:29 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:29 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:29.852458 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.847198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:47883)
0807 15:15:29.847477 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.847491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.847592 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.847833 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.847835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.848021 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:29.848121 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.848126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.848850 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.848852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.849431 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.849439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.849903 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.849968 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.850460 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:29.850471 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:29.852326 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
W20260807 15:15:29.852591 13198 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:29.847259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:47883 (local address 127.31.230.254:38555)
0807 15:15:29.847417 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.847421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.847557 (+ 136us) server_negotiation.cc:407] Connection header received
0807 15:15:29.847671 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.847673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.847721 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.847795 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.848235 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.848755 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849551 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.849802 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849882 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.852464 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:47883: BlockingRecv error: recv got EOF from 127.31.230.194:47883 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:15:30 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:30 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:30.052996 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:30.047635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:50527)
0807 15:15:30.047910 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:30.047923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:30.048028 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:30.048274 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:30.048276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:30.048460 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:30.048560 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:30.048566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.049271 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.049273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:30.049880 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:30.049889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.050302 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.050304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:30.050368 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:30.050937 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:30.050950 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:30.052865 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":99,"threads_started":1}
W20260807 15:15:30.053190 13198 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:30.047797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:50527 (local address 127.31.230.254:38555)
0807 15:15:30.047938 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:30.047942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:30.047977 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:15:30.048110 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:30.048112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:30.048160 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:30.048236 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:30.048662 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.049176 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.049982 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.050200 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.050293 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:30.053044 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:50527: BlockingRecv error: recv got EOF from 127.31.230.195:50527 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 15:15:30 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:30 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:30.663406 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:30.657732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:46931)
0807 15:15:30.658088 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:30.658109 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:30.658282 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:30.658559 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:30.658562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:30.658769 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:30.658882 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:30.658887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.659643 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.659646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:30.660264 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:30.660272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.660694 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.660696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:30.660764 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:30.661285 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:30.661297 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:30.663236 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
W20260807 15:15:30.663528 13203 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:30.657790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46931 (local address 127.31.230.254:38555)
0807 15:15:30.658079 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:30.658084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:30.658239 (+ 155us) server_negotiation.cc:407] Connection header received
0807 15:15:30.658379 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:30.658382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:30.658431 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:30.658515 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:30.658991 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.659544 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.660369 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.660591 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.660668 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:30.663394 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:46931: BlockingRecv error: recv got EOF from 127.31.230.193:46931 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":71,"threads_started":1}
Aug 07 15:15:30 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:30 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:30.859018 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:30.853398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:34433)
0807 15:15:30.853703 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:30.853717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:30.853820 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:30.854066 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:30.854068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:30.854266 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:30.854363 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:30.854369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.855154 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.855158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:30.855792 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:30.855802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.856262 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.856265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:30.856340 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:30.856870 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:30.856883 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:30.858898 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":140,"threads_started":1}
W20260807 15:15:30.859157 13203 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:30.853439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:34433 (local address 127.31.230.254:38555)
0807 15:15:30.853615 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:30.853620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:30.853785 (+ 165us) server_negotiation.cc:407] Connection header received
0807 15:15:30.853896 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:30.853898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:30.853946 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:30.854031 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:30.854465 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.855010 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.855919 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:30.856150 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:30.856253 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:30.859039 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:34433: BlockingRecv error: recv got EOF from 127.31.230.194:34433 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 15:15:31 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:31 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:31.059520 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.054037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:43389)
0807 15:15:31.054391 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.054408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.054535 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.054738 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.054740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.054927 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:31.055027 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.055033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.055770 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.055772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.056354 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.056363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.056822 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.056824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.056890 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.057456 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:31.057475 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:31.059400 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":116,"threads_started":1}
W20260807 15:15:31.059772 13203 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:31.054096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:43389 (local address 127.31.230.254:38555)
0807 15:15:31.054235 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.054265 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.054491 (+ 226us) server_negotiation.cc:407] Connection header received
0807 15:15:31.054604 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.054606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.054659 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.054726 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.055124 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.055663 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.056461 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.056706 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.056784 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.059544 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:43389: BlockingRecv error: recv got EOF from 127.31.230.195:43389 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 15:15:31 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:31 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:31.669976 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.664361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:40287)
0807 15:15:31.664653 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.664667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.664758 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.664983 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.664985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.665175 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:31.665304 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.665310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.666080 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.666082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.666679 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.666689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.667125 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.667128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.667193 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.667783 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:31.667797 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:31.669811 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":113,"threads_started":1}
W20260807 15:15:31.670121 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:31.664474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:40287 (local address 127.31.230.254:38555)
0807 15:15:31.664740 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.664765 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.664779 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:31.664818 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.664820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.664862 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.664944 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.665416 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.665989 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.666800 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.667027 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.667101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.669970 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:40287: BlockingRecv error: recv got EOF from 127.31.230.193:40287 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":115,"threads_started":1}
Aug 07 15:15:31 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:31 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:31.865275 13210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.859857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:46861)
0807 15:15:31.860136 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.860150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.860246 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.860487 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.860490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.860685 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:31.860799 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.860805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.861660 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.861663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.862268 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.862278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.862679 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.862681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.862755 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.863252 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:31.863264 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:31.865155 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
W20260807 15:15:31.865463 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:31.860032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:46861 (local address 127.31.230.254:38555)
0807 15:15:31.860166 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.860170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.860201 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:15:31.860330 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.860332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.860378 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.860453 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.860897 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.861560 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.862371 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.862585 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.862661 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.865296 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:46861: BlockingRecv error: recv got EOF from 127.31.230.194:46861 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 15:15:32 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:32 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:32.066645 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.060508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:38495)
0807 15:15:32.060870 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.060884 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.061020 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.061316 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.061319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.061617 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:32.061782 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.061795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.062648 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.062651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.063254 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.063263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.063745 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.063749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.063815 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.064400 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:32.064412 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:32.066529 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
W20260807 15:15:32.066864 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:32.060636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:38495 (local address 127.31.230.254:38555)
0807 15:15:32.060801 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.060808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.060974 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:15:32.061131 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.061134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.061188 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.061273 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.061926 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.062520 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.063379 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.063637 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.063729 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.066713 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:38495: BlockingRecv error: recv got EOF from 127.31.230.195:38495 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 15:15:32 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:32 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:32.677042 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.671029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:59901)
0807 15:15:32.671330 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.671344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.671441 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.671683 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.671685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.671897 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:32.671997 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.672003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.672761 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.672763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.673363 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.673374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.674003 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.674007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.674100 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.674879 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:32.674894 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:32.676926 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
W20260807 15:15:32.677249 13214 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:32.671098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:59901 (local address 127.31.230.254:38555)
0807 15:15:32.671380 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.671385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.671399 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:32.671525 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.671527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.671573 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.671651 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.672097 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.672634 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.673480 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.673877 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.673983 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.677068 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:59901: BlockingRecv error: recv got EOF from 127.31.230.193:59901 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
Aug 07 15:15:32 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:32 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:32.871519 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.866091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:52589)
0807 15:15:32.866392 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.866406 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.866510 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.866800 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.866802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.866997 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:32.867094 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.867105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.867905 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.867907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.868461 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.868468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.868952 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.868954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.869013 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.869519 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:32.869557 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:32.871389 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":134,"threads_started":1}
W20260807 15:15:32.871658 13214 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:32.866148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:52589 (local address 127.31.230.254:38555)
0807 15:15:32.866315 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.866319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.866475 (+ 156us) server_negotiation.cc:407] Connection header received
0807 15:15:32.866643 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.866646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.866702 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.866788 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.867213 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.867801 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.868576 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.868840 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.868937 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.871537 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:52589: BlockingRecv error: recv got EOF from 127.31.230.194:52589 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 15:15:33 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:33 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:33.074514 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:33.067605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:40595)
0807 15:15:33.067999 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:33.068025 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:33.068202 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:33.068541 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:33.068545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:33.068794 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:33.068910 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:33.068918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.069904 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.069912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:33.070564 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:33.070574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.071126 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.071131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:33.071251 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:33.071906 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:33.071919 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:33.074318 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":126,"threads_started":1}
W20260807 15:15:33.074641 13214 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:33.067745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:40595 (local address 127.31.230.254:38555)
0807 15:15:33.067958 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:33.067965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:33.068125 (+ 160us) server_negotiation.cc:407] Connection header received
0807 15:15:33.068325 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:33.068329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:33.068395 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:33.068504 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:33.069072 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.069773 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.070712 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.070972 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.071126 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:33.074525 (+ 3399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:40595: BlockingRecv error: recv got EOF from 127.31.230.195:40595 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 15:15:33 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:33 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:33.683465 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:33.677904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:34403)
0807 15:15:33.678227 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:33.678241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:33.678328 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:33.678577 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:33.678579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:33.678776 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:33.678879 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:33.678885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.679632 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.679634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:33.680216 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:33.680246 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.680761 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.680763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:33.680828 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:33.681399 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:33.681421 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:33.683344 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":98,"threads_started":1}
W20260807 15:15:33.683691 13222 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:33.678022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:34403 (local address 127.31.230.254:38555)
0807 15:15:33.678264 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:33.678268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:33.678303 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:15:33.678416 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:33.678419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:33.678466 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:33.678545 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:33.678974 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.679534 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.680367 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.680612 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.680697 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:33.683526 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:34403: BlockingRecv error: recv got EOF from 127.31.230.193:34403 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
Aug 07 15:15:33 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:33 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:33.882355 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:33.872530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:41153)
0807 15:15:33.872885 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:33.872900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:33.873014 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:33.873297 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:33.873299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:33.873513 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:33.873639 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:33.873645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.874504 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.874509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:33.875247 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:33.875275 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.875906 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.875910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:33.876016 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:33.879956 (+ 3940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:33.879968 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:33.882230 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":110,"threads_started":1}
W20260807 15:15:33.882671 13222 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:33.872598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:41153 (local address 127.31.230.254:38555)
0807 15:15:33.872775 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:33.872780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:33.872983 (+ 203us) server_negotiation.cc:407] Connection header received
0807 15:15:33.873128 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:33.873131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:33.873183 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:33.873264 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:33.873752 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.874346 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.875390 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:33.875773 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:33.875917 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:33.882391 (+ 6474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:41153: BlockingRecv error: recv got EOF from 127.31.230.194:41153 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 15:15:34 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:34 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:34.081821 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.075491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:49897)
0807 15:15:34.075886 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:34.075907 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:34.076060 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:34.076384 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:34.076387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:34.076643 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:34.076848 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.076857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.077796 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.077801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.078432 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.078441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.078944 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.078948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.079044 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.079638 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:34.079652 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:34.081690 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":101,"threads_started":1}
W20260807 15:15:34.081969 13222 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:34.075627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:49897 (local address 127.31.230.254:38555)
0807 15:15:34.075819 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:34.075825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:34.075991 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:15:34.076206 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:34.076209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:34.076263 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:34.076358 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:34.077002 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.077680 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.078564 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.078820 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.078940 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.081847 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:49897: BlockingRecv error: recv got EOF from 127.31.230.195:49897 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:15:34 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:34 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:34.689671 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.684274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:49035)
0807 15:15:34.684581 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:34.684598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:34.684721 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:34.684975 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:34.684977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:34.685163 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:34.685266 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.685271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.686017 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.686020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.686582 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.686590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.686997 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.687000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.687064 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.687622 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:34.687644 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:34.689491 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
W20260807 15:15:34.689816 13227 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:34.684427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:49035 (local address 127.31.230.254:38555)
0807 15:15:34.684680 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:34.684683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:34.684697 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:34.684809 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:34.684812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:34.684858 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:34.684937 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:34.685367 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.685918 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.686683 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.686903 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.686978 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.689656 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:49035: BlockingRecv error: recv got EOF from 127.31.230.193:49035 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
Aug 07 15:15:34 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:34 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:34.889191 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.883133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:58327)
0807 15:15:34.883449 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:34.883462 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:34.883589 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:34.883868 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:34.883871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:34.884090 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:34.884186 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.884192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.885107 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.885112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.885731 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.885739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.886208 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.886212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.886304 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.886899 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:34.886914 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:34.889073 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":111,"threads_started":1}
W20260807 15:15:34.889431 13227 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:34.883197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:58327 (local address 127.31.230.254:38555)
0807 15:15:34.883376 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:34.883381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:34.883536 (+ 155us) server_negotiation.cc:407] Connection header received
0807 15:15:34.883692 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:34.883697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:34.883750 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:34.883839 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:34.884322 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.884995 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.885860 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.886091 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.886201 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.889216 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:58327: BlockingRecv error: recv got EOF from 127.31.230.194:58327 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 15:15:35 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:35 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:35.088543 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.082623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:36999)
0807 15:15:35.082936 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.082948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.083087 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.083370 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.083374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.083591 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.083691 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.083697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.084471 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.084476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.085161 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.085171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.085782 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.085786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.085885 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.086515 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:35.086527 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.088420 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":94,"threads_started":1}
W20260807 15:15:35.088685 13227 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:35.082702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:36999 (local address 127.31.230.254:38555)
0807 15:15:35.082875 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.082881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.083027 (+ 146us) server_negotiation.cc:407] Connection header received
0807 15:15:35.083191 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.083194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.083250 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.083339 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.083814 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.084356 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.085301 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.085657 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.085774 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.088563 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:36999: BlockingRecv error: recv got EOF from 127.31.230.195:36999 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 15:15:35 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:35 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:35.696024 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.690494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:56961)
0807 15:15:35.690786 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.690801 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.690894 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.691148 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.691150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.691342 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.691446 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.691452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.692195 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.692197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.692757 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.692765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.693175 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.693178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.693240 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.693740 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:35.693763 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.695853 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
W20260807 15:15:35.696143 13231 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:35.690578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:56961 (local address 127.31.230.254:38555)
0807 15:15:35.690828 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.690832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.690845 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:35.690980 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.690982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.691029 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.691110 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.691550 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.692101 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.692858 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.693080 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.693156 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.696000 (+ 2844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:56961: BlockingRecv error: recv got EOF from 127.31.230.193:56961 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":83,"threads_started":1}
Aug 07 15:15:35 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:35 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:35.895833 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.889961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:37833)
0807 15:15:35.890302 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.890320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.890472 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.890764 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.890767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.890983 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.891089 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.891094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.891874 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.891879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.892444 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.892452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.892914 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.892917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.893007 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.893655 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:35.893668 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.895642 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
W20260807 15:15:35.895898 13231 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:35.890121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:37833 (local address 127.31.230.254:38555)
0807 15:15:35.890304 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.890311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.890408 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:15:35.890574 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.890578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.890630 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.890719 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.891216 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.891760 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.892566 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.892796 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.892907 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.895779 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:37833: BlockingRecv error: recv got EOF from 127.31.230.194:37833 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 15:15:36 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:36 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:36.094889 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.089339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:55179)
0807 15:15:36.089684 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:36.089698 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:36.089816 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:36.090054 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:36.090056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:36.090245 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:36.090346 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.090351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.091091 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.091093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.091652 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.091659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.092074 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.092077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.092143 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.092720 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:36.092734 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:36.094702 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":101,"threads_started":1}
W20260807 15:15:36.094929 13231 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:36.089423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:55179 (local address 127.31.230.254:38555)
0807 15:15:36.089594 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:36.089599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:36.089782 (+ 183us) server_negotiation.cc:407] Connection header received
0807 15:15:36.089892 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:36.089894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:36.089943 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:36.090018 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:36.090463 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.090992 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.091754 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.091973 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.092050 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.094828 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:55179: BlockingRecv error: recv got EOF from 127.31.230.195:55179 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 15:15:36 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:36 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:36.702414 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.696908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:59889)
0807 15:15:36.697197 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:36.697211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:36.697314 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:36.697674 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:36.697676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:36.697859 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:36.697957 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.697963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.698700 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.698703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.699275 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.699283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.699768 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.699771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.699833 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.700332 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:36.700343 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:36.702296 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
W20260807 15:15:36.702741 13236 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:36.697002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:59889 (local address 127.31.230.254:38555)
0807 15:15:36.697412 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:36.697416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:36.697430 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:36.697473 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:36.697475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:36.697518 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:36.697639 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:36.698060 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.698600 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.699405 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.699661 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.699754 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.702585 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:59889: BlockingRecv error: recv got EOF from 127.31.230.193:59889 (error 108)
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":148,"threads_started":1}
Aug 07 15:15:36 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:36 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:36.902689 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.896750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:45391)
0807 15:15:36.897115 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:36.897130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:36.897228 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:36.897589 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:36.897596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:36.897798 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:36.897899 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.897904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.898728 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.898731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.899337 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.899345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.899801 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.899803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.899864 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.900383 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:36.900396 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:36.902509 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
W20260807 15:15:36.902756 13236 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:36.896862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:45391 (local address 127.31.230.254:38555)
0807 15:15:36.897062 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:36.897066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:36.897203 (+ 137us) server_negotiation.cc:407] Connection header received
0807 15:15:36.897314 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:36.897316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:36.897364 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:36.897459 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:36.898057 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.898617 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.899460 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.899697 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.899789 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.902644 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:45391: BlockingRecv error: recv got EOF from 127.31.230.194:45391 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 15:15:37 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:37 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:37.101351 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:37.095813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:58127)
0807 15:15:37.096117 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:37.096130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:37.096232 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:37.096464 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:37.096466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:37.096658 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:37.096778 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:37.096785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.097613 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.097618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:37.098200 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:37.098209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.098672 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.098676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:37.098766 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.099244 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:37.099256 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:37.101158 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":123,"threads_started":1}
W20260807 15:15:37.101389 13236 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:37.095968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:58127 (local address 127.31.230.254:38555)
0807 15:15:37.096104 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:37.096108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:37.096204 (+ 96us) server_negotiation.cc:407] Connection header received
0807 15:15:37.096311 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:37.096314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:37.096360 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:37.096428 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:37.096907 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.097453 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.098327 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.098555 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.098674 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.101277 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:58127: BlockingRecv error: recv got EOF from 127.31.230.195:58127 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 15:15:37 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:37 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:37.708904 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:37.703222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:58971)
0807 15:15:37.703491 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:37.703506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:37.703611 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:37.703865 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:37.703867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:37.704045 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:37.704146 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:37.704151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.704916 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.704918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:37.705483 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:37.705491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.706009 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.706011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:37.706078 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.706570 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:37.706582 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:37.708710 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
W20260807 15:15:37.709000 13240 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:37.703276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:58971 (local address 127.31.230.254:38555)
0807 15:15:37.703511 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:37.703515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:37.703572 (+ 57us) server_negotiation.cc:407] Connection header received
0807 15:15:37.703690 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:37.703693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:37.703738 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:37.703826 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:37.704271 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.704825 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.705677 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.705909 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.705987 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.708855 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:58971: BlockingRecv error: recv got EOF from 127.31.230.193:58971 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
Aug 07 15:15:37 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:37 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:37.909287 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:37.903723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:34931)
0807 15:15:37.904009 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:37.904023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:37.904134 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:37.904380 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:37.904382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:37.904569 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:37.904667 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:37.904673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.905451 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.905455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:37.906188 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:37.906199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.906760 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.906763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:37.906830 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.907331 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:37.907342 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:37.909166 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
W20260807 15:15:37.909433 13240 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:37.903777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:34931 (local address 127.31.230.254:38555)
0807 15:15:37.903931 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:37.903936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:37.904094 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:15:37.904213 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:37.904216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:37.904274 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:37.904343 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:37.904760 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.905316 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.906327 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.906650 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.906732 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.909313 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:34931: BlockingRecv error: recv got EOF from 127.31.230.194:34931 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 15:15:38 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:38 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:38.108136 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:38.102369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:56135)
0807 15:15:38.102661 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:38.102674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:38.102786 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:38.103022 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:38.103024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:38.103238 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:38.103340 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:38.103347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.104059 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.104061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:38.104623 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:38.104631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.105033 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.105035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:38.105100 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:38.105736 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:38.105782 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:38.107958 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":112,"threads_started":1}
W20260807 15:15:38.108199 13240 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:38.102447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:56135 (local address 127.31.230.254:38555)
0807 15:15:38.102593 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:38.102597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:38.102747 (+ 150us) server_negotiation.cc:407] Connection header received
0807 15:15:38.102858 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:38.102860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:38.102911 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:38.102984 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:38.103441 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.103963 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.104720 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.104937 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.105011 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:38.108086 (+ 3075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:56135: BlockingRecv error: recv got EOF from 127.31.230.195:56135 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 15:15:38 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:38 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:38.715377 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:38.709795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:44873)
0807 15:15:38.710065 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:38.710078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:38.710179 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:38.710448 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:38.710450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:38.710637 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:38.710737 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:38.710759 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.711570 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.711573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:38.712158 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:38.712168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.712627 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.712630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:38.712695 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:38.713183 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:38.713195 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:38.715223 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
W20260807 15:15:38.715584 13246 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:38.709851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:44873 (local address 127.31.230.254:38555)
0807 15:15:38.710089 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:38.710093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:38.710132 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:15:38.710282 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:38.710285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:38.710333 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:38.710414 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:38.710856 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.711429 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.712285 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.712512 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.712593 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:38.715416 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:44873: BlockingRecv error: recv got EOF from 127.31.230.193:44873 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
Aug 07 15:15:38 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:38 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:38.919386 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:38.910098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:57359)
0807 15:15:38.910387 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:38.910400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:38.910494 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:38.910789 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:38.910791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:38.910979 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:38.911079 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:38.911084 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.911835 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.911837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:38.912395 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:38.912402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.912811 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.912813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:38.912875 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:38.917280 (+ 4405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:38.917293 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:38.919259 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":110,"threads_started":1}
W20260807 15:15:38.919586 13246 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:38.910316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:57359 (local address 127.31.230.254:38555)
0807 15:15:38.910453 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:38.910457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:38.910471 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:38.910580 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:38.910583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:38.910630 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:38.910707 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:38.911180 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.911722 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.912494 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:38.912710 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:38.912785 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:38.919422 (+ 6637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:57359: BlockingRecv error: recv got EOF from 127.31.230.194:57359 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 15:15:39 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:39 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:39.114797 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:39.108925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:35737)
0807 15:15:39.109276 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:39.109290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:39.109440 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:39.109755 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:39.109759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:39.109977 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:39.110078 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:39.110084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.110890 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.110895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:39.111478 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:39.111487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.111984 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.111988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:39.112079 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.112593 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:39.112604 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:39.114618 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":89,"threads_started":1}
W20260807 15:15:39.114892 13246 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:39.109053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:35737 (local address 127.31.230.254:38555)
0807 15:15:39.109221 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:39.109226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:39.109374 (+ 148us) server_negotiation.cc:407] Connection header received
0807 15:15:39.109567 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:39.109571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:39.109625 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:39.109731 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:39.110205 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.110772 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.111609 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.111865 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.111979 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.114764 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:35737: BlockingRecv error: recv got EOF from 127.31.230.195:35737 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 15:15:39 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:39 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:39.722291 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:39.716343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:55271)
0807 15:15:39.716701 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:39.716720 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:39.716892 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:39.717235 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:39.717237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:39.717451 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:39.717641 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:39.717651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.718465 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.718468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:39.719036 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:39.719045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.719514 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.719517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:39.719584 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.720204 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:39.720218 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:39.722157 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":92,"threads_started":1}
W20260807 15:15:39.722462 13252 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:39.716421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:55271 (local address 127.31.230.254:38555)
0807 15:15:39.716745 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:39.716749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:39.716820 (+ 71us) server_negotiation.cc:407] Connection header received
0807 15:15:39.717017 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:39.717021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:39.717077 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:39.717165 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:39.717776 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.718345 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.719152 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.719410 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.719492 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.722322 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:55271: BlockingRecv error: recv got EOF from 127.31.230.193:55271 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":113,"threads_started":1}
Aug 07 15:15:39 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:39 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:39.925904 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:39.920275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:55657)
0807 15:15:39.920576 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:39.920590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:39.920693 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:39.921007 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:39.921009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:39.921210 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:39.921320 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:39.921326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.922176 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.922179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:39.922747 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:39.922755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.923195 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.923197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:39.923261 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.923780 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:39.923799 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:39.925784 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
W20260807 15:15:39.926147 13252 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:39.920413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:55657 (local address 127.31.230.254:38555)
0807 15:15:39.920586 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:39.920590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:39.920662 (+ 72us) server_negotiation.cc:407] Connection header received
0807 15:15:39.920795 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:39.920797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:39.920845 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:39.920920 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:39.921445 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.922073 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.922862 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.923090 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.923171 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.925924 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:55657: BlockingRecv error: recv got EOF from 127.31.230.194:55657 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 15:15:40 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:40 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:40.121726 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:40.115785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:50441)
0807 15:15:40.116058 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:40.116072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:40.116199 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:40.116462 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:40.116464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:40.116660 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:40.116782 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:40.116788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.117682 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.117685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:40.118255 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:40.118264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.118768 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.118772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:40.118866 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:40.119453 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:40.119468 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:40.121520 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
W20260807 15:15:40.121989 13252 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:40.115848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:50441 (local address 127.31.230.254:38555)
0807 15:15:40.115993 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:40.115997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:40.116166 (+ 169us) server_negotiation.cc:407] Connection header received
0807 15:15:40.116293 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:40.116296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:40.116349 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:40.116419 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:40.116919 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.117476 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.118385 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.118618 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.118720 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:40.121809 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:50441: BlockingRecv error: recv got EOF from 127.31.230.195:50441 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 15:15:40 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:40 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:40.728636 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:40.723060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:49013)
0807 15:15:40.723414 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:40.723429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:40.723530 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:40.723760 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:40.723762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:40.723956 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:40.724104 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:40.724111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.724871 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.724874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:40.725474 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:40.725484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.725942 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.725945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:40.726008 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:40.726534 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:40.726546 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:40.728458 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":98,"threads_started":1}
W20260807 15:15:40.728720 13256 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:40.723194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:49013 (local address 127.31.230.254:38555)
0807 15:15:40.723542 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:40.723545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:40.723558 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:15:40.723596 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:40.723598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:40.723639 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:40.723717 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:40.724215 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.724765 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.725605 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.725844 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.725921 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:40.728589 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:49013: BlockingRecv error: recv got EOF from 127.31.230.193:49013 (error 108)
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":70,"threads_started":1}
Aug 07 15:15:40 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:40 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:40.932623 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:40.926736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:58435)
0807 15:15:40.927043 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:40.927057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:40.927161 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:40.927405 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:40.927407 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:40.927607 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:40.927721 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:40.927727 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.928450 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.928452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:40.929022 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:40.929030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.929479 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.929483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:40.929605 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:40.930236 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:40.930251 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:40.932446 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":114,"threads_started":1}
W20260807 15:15:40.932683 13256 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:40.926844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:58435 (local address 127.31.230.254:38555)
0807 15:15:40.926999 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:40.927003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:40.927126 (+ 123us) server_negotiation.cc:407] Connection header received
0807 15:15:40.927240 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:40.927243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:40.927290 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:40.927365 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:40.927825 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.928354 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.929121 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:40.929337 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:40.929430 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:40.932573 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:58435: BlockingRecv error: recv got EOF from 127.31.230.194:58435 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:15:41 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:41 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.195@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:41.128376 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:41.122668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.195:60741)
0807 15:15:41.122990 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:41.123004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:41.123110 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:41.123365 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:41.123367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:41.123570 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:41.123675 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:41.123680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.124441 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.124444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:41.125001 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:41.125009 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.125429 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.125431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:41.125496 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:41.126017 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:41.126029 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:41.128183 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":115,"threads_started":1}
W20260807 15:15:41.128451 13256 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:41.122761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:60741 (local address 127.31.230.254:38555)
0807 15:15:41.122922 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:41.122926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:41.123078 (+ 152us) server_negotiation.cc:407] Connection header received
0807 15:15:41.123191 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:41.123193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:41.123245 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:41.123320 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:41.123809 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.124343 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.125106 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.125327 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.125404 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:41.128325 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.195:60741: BlockingRecv error: recv got EOF from 127.31.230.195:60741 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 15:15:41 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:41 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.193@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:41.736611 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:41.729848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.193:36297)
0807 15:15:41.730187 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:41.730202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:41.730309 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:41.730561 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:41.730563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:41.730772 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:41.730876 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:41.730882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.731793 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.731796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:41.732458 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:41.732468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.733134 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.733140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:41.733239 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:41.734316 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:41.734337 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:41.736473 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":130,"threads_started":1}
W20260807 15:15:41.736784 13263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:41.729998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:36297 (local address 127.31.230.254:38555)
0807 15:15:41.730302 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:41.730306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:41.730322 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:15:41.730375 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:41.730378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:41.730418 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:41.730514 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:41.731033 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.731633 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.732639 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.732979 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.733093 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:41.736627 (+ 3534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.193:36297: BlockingRecv error: recv got EOF from 127.31.230.193:36297 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":141,"threads_started":1}
Aug 07 15:15:41 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:15:41 dist-test-slave-gkw7 krb5kdc[11774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.230.194@KRBTEST.COM for oryx/127.31.230.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:15:41.940119 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:41.933994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:38555 (local address 127.31.230.194:40005)
0807 15:15:41.934355 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:41.934371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:41.934496 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:41.934845 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:41.934849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:41.935097 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:41.935240 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:41.935248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.936178 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.936181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:41.936760 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:41.936768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.937237 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.937240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:41.937304 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:41.937842 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:41.937854 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:41.939946 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.230.254:38555: Server oryx/127.31.230.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":105,"threads_started":1}
W20260807 15:15:41.940188 13263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:41.934139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:40005 (local address 127.31.230.254:38555)
0807 15:15:41.934339 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:41.934342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:41.934457 (+ 115us) server_negotiation.cc:407] Connection header received
0807 15:15:41.934620 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:41.934624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:41.934680 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:41.934787 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:41.935397 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.936056 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.936889 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:41.937117 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:41.937209 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:41.940093 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.230.194:40005: BlockingRecv error: recv got EOF from 127.31.230.194:40005 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 15:15:42.016778 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 12605
I20260807 15:15:42.030637 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 12823
I20260807 15:15:42.044121 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 13002
I20260807 15:15:42.052382 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 12510
2026-08-07T15:15:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18376 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T15:15:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:42Z Disabled control of system clock
I20260807 15:15:42.089587 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:43855
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:43855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:15:42.205260 13271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:42.205590 13271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:42.205639 13271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:42.209590 13271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:42.209645 13271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:42.209661 13271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:42.209676 13271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:42.209692 13271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:42.214402 13271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:43855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:43855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:42.215514 13271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:42.216854 13271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:42.226106 13271 server_base.cc:1034] running on GCE node
W20260807 15:15:42.226122 13279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.226122 13281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.226054 13278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:42.227070 13271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:42.228374 13271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:42.229636 13271 hybrid_clock.cc:648] HybridClock initialized: now 1786115742229607 us; error 37 us; skew 500 ppm
I20260807 15:15:42.231580 13271 webserver.cc:460] Webserver started at http://127.31.230.254:46663/ using document root <none> and password file <none>
I20260807 15:15:42.232218 13271 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:42.232321 13271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:42.232563 13271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:42.234894 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "10daf7e16daa441face8a9e386a551ff"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "bdfb34b0d8f4f5b2275ce2c8e1bfe813"
server_key_iv: "a2519fd43179a36cc8becb82c44508cc"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.235491 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "10daf7e16daa441face8a9e386a551ff"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "bdfb34b0d8f4f5b2275ce2c8e1bfe813"
server_key_iv: "a2519fd43179a36cc8becb82c44508cc"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.239427 13271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:15:42.241951 13290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.243211 13271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:42.243361 13271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "10daf7e16daa441face8a9e386a551ff"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "bdfb34b0d8f4f5b2275ce2c8e1bfe813"
server_key_iv: "a2519fd43179a36cc8becb82c44508cc"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.243492 13271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:42.266281 13271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:42.267015 13271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:42.267225 13271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:42.274356 13271 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:43855
I20260807 15:15:42.274382 13358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:43855 every 8 connection(s)
I20260807 15:15:42.275324 13271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:42.275949 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 13271
I20260807 15:15:42.276093 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:42.276391 32667 external_mini_cluster.cc:1428] Setting key 97d11e9af2dedf980d76c8e2cb95c239
I20260807 15:15:42.279026 13359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:42.285705 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff: Bootstrap starting.
I20260807 15:15:42.286039 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.277915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41612 (local address 127.31.230.254:43855)
0807 15:15:42.278551 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.278571 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.278609 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:15:42.279320 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.279339 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.279737 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.280020 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.280377 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.281330 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.282625 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.283085 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.283214 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.284156 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:42.284161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:42.284173 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:42.284565 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:42.284619 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:42.284640 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:42.284721 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:42.284907 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 15:15:42.285230 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":151,"threads_started":1}
I20260807 15:15:42.288277 13359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:42.289064 13359 log.cc:824] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:42.291030 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff: No bootstrap required, opened a new log
I20260807 15:15:42.294623 13359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10daf7e16daa441face8a9e386a551ff" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43855 } }
I20260807 15:15:42.294839 13359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:42.294915 13359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10daf7e16daa441face8a9e386a551ff, State: Initialized, Role: FOLLOWER
I20260807 15:15:42.295326 13359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10daf7e16daa441face8a9e386a551ff" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43855 } }
I20260807 15:15:42.295475 13359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:42.295537 13359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:42.295662 13359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:42.296568 13359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10daf7e16daa441face8a9e386a551ff" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43855 } }
I20260807 15:15:42.296911 13359 leader_election.cc:302] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10daf7e16daa441face8a9e386a551ff; no voters:
I20260807 15:15:42.297224 13359 leader_election.cc:288] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:42.297358 13365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:42.297665 13365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [term 1 LEADER]: Becoming Leader. State: Replica: 10daf7e16daa441face8a9e386a551ff, State: Running, Role: LEADER
I20260807 15:15:42.298040 13365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10daf7e16daa441face8a9e386a551ff" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43855 } }
I20260807 15:15:42.298554 13359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:42.299791 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10daf7e16daa441face8a9e386a551ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10daf7e16daa441face8a9e386a551ff" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43855 } } }
I20260807 15:15:42.299791 13367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10daf7e16daa441face8a9e386a551ff. Latest consensus state: current_term: 1 leader_uuid: "10daf7e16daa441face8a9e386a551ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10daf7e16daa441face8a9e386a551ff" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 43855 } } }
I20260807 15:15:42.300032 13367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:42.300032 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff [sys.catalog]: This master's current role is: LEADER
W20260807 15:15:42.303499 13388 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:42.303570 13388 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:42.303655 13377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:42.304342 13377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:42.309885 13377 catalog_manager.cc:1347] Generated new cluster ID: 71e2ff29e668458fa6051218c3f2f635
I20260807 15:15:42.309948 13377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:42.318120 13377 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:42.319068 13377 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:42.328450 13377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10daf7e16daa441face8a9e386a551ff: Generated new TSK 0
I20260807 15:15:42.329159 13377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:42.332701 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43855
--builtin_ntp_servers=127.31.230.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:15:42.451386 13393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:42.451700 13393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:42.451783 13393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:42.456507 13393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:42.456594 13393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:42.456686 13393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:42.462069 13393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:42.463285 13393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:42.464761 13393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:42.474591 13403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.474597 13399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.474660 13398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:42.474869 13393 server_base.cc:1034] running on GCE node
I20260807 15:15:42.475293 13393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:42.475930 13393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:42.477118 13393 hybrid_clock.cc:648] HybridClock initialized: now 1786115742477082 us; error 49 us; skew 500 ppm
I20260807 15:15:42.479005 13393 webserver.cc:460] Webserver started at http://127.31.230.193:41299/ using document root <none> and password file <none>
I20260807 15:15:42.479734 13393 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:42.479830 13393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:42.480055 13393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:42.482292 13393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "076fe03fc9424421bcd3445026917804"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "9d6c074ace734c4c62fe5da1a63ff71f"
server_key_iv: "50ba466f434782c1f71d255d2c3dff25"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.482884 13393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "076fe03fc9424421bcd3445026917804"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "9d6c074ace734c4c62fe5da1a63ff71f"
server_key_iv: "50ba466f434782c1f71d255d2c3dff25"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.486872 13393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:42.489645 13413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.490921 13393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:42.491022 13393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "076fe03fc9424421bcd3445026917804"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "9d6c074ace734c4c62fe5da1a63ff71f"
server_key_iv: "50ba466f434782c1f71d255d2c3dff25"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.491114 13393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:42.503146 13393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:42.503786 13393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:42.503937 13393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:42.504467 13393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:42.505360 13393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:42.505409 13393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.505450 13393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:42.505509 13393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.514446 13393 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:35529
I20260807 15:15:42.514480 13571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:35529 every 8 connection(s)
I20260807 15:15:42.515384 13393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:42.520417 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 13393
I20260807 15:15:42.520515 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:42.520793 32667 external_mini_cluster.cc:1428] Setting key b7462d60e459666648d4778b8c15dd35
I20260807 15:15:42.522802 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43855
--builtin_ntp_servers=127.31.230.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 15:15:42.525617 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.517196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:52943 (local address 127.31.230.254:43855)
0807 15:15:42.517345 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.517349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.518503 (+ 1154us) server_negotiation.cc:407] Connection header received
0807 15:15:42.520040 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.520042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.520092 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.520212 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.521229 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.521852 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.523265 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.523675 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.523799 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.524969 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:42.524975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:42.524980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:42.524997 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:42.525025 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:42.525038 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:42.525126 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:42.525283 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:15:42.525383 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:15:42.526448 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.517650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43855 (local address 127.31.230.193:52943)
0807 15:15:42.518328 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:42.518369 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:42.519848 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:42.520369 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:42.520376 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:42.520457 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:42.521057 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:42.521072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.521988 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.521991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:42.523132 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:42.523144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.523837 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.523839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:42.523982 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.524669 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:42.524673 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:42.524739 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:42.524807 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:42.525133 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:42.525140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:42.525145 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:42.525252 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:15:42.525487 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":125,"threads_started":1}
I20260807 15:15:42.527850 13572 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43855
I20260807 15:15:42.528110 13572 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:42.528657 13572 heartbeater.cc:511] Master 127.31.230.254:43855 requested a full tablet report, sending...
I20260807 15:15:42.530485 13317 ts_manager.cc:194] Registered new tserver with Master: 076fe03fc9424421bcd3445026917804 (127.31.230.193:35529)
I20260807 15:15:42.532301 13317 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.193:52943
W20260807 15:15:42.637737 13575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:42.638002 13575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:42.638039 13575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:42.641940 13575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:42.641993 13575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:42.642045 13575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:42.646777 13575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:42.648003 13575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:42.649441 13575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:42.659287 13575 server_base.cc:1034] running on GCE node
W20260807 15:15:42.659299 13588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.659298 13587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.659452 13590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:42.659888 13575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:42.660516 13575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:42.661706 13575 hybrid_clock.cc:648] HybridClock initialized: now 1786115742661696 us; error 33 us; skew 500 ppm
I20260807 15:15:42.663491 13575 webserver.cc:460] Webserver started at http://127.31.230.194:35149/ using document root <none> and password file <none>
I20260807 15:15:42.664086 13575 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:42.664180 13575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:42.664425 13575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:42.666540 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "00e79da9b4944a39b685042d3f11615a"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "31675bf92d8827635bb6f835ca8d546e"
server_key_iv: "8970423c62802acc076a8dddd9253aaa"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.667137 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "00e79da9b4944a39b685042d3f11615a"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "31675bf92d8827635bb6f835ca8d546e"
server_key_iv: "8970423c62802acc076a8dddd9253aaa"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.670998 13575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:15:42.673646 13596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.674894 13575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:42.675043 13575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "00e79da9b4944a39b685042d3f11615a"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "31675bf92d8827635bb6f835ca8d546e"
server_key_iv: "8970423c62802acc076a8dddd9253aaa"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.675166 13575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:42.692070 13575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:42.692821 13575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:42.693025 13575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:42.693713 13575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:42.694718 13575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:42.694798 13575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.694876 13575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:42.694929 13575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.704171 13575 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:39401
I20260807 15:15:42.704193 13771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:39401 every 8 connection(s)
I20260807 15:15:42.705128 13575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:42.710736 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 13575
I20260807 15:15:42.710867 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:42.711171 32667 external_mini_cluster.cc:1428] Setting key 1b4d71d307a20d49719cd21fe0a77e44
I20260807 15:15:42.713346 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43855
--builtin_ntp_servers=127.31.230.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 15:15:42.714844 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.706998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:35699 (local address 127.31.230.254:43855)
0807 15:15:42.707141 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.707144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.708237 (+ 1093us) server_negotiation.cc:407] Connection header received
0807 15:15:42.709755 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.709758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.709826 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.709926 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.710863 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.711488 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.712718 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.712945 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.713027 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.714371 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:42.714373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:42.714375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:42.714387 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:42.714409 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:42.714416 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:42.714472 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:42.714662 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 15:15:42.714720 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:15:42.716001 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.707424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43855 (local address 127.31.230.194:35699)
0807 15:15:42.708084 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:42.708125 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:42.709459 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:42.710117 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:42.710124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:42.710210 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:42.710717 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:42.710729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.711638 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.711641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:42.712584 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:42.712599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.713114 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.713121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:42.713364 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.714040 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:42.714045 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:42.714105 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:42.714193 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:42.714501 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:42.714513 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:42.714522 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:15:42.714657 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 15:15:42.715077 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260807 15:15:42.717196 13773 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43855
I20260807 15:15:42.717464 13773 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:42.718017 13773 heartbeater.cc:511] Master 127.31.230.254:43855 requested a full tablet report, sending...
I20260807 15:15:42.719094 13317 ts_manager.cc:194] Registered new tserver with Master: 00e79da9b4944a39b685042d3f11615a (127.31.230.194:39401)
I20260807 15:15:42.719668 13317 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.194:35699
W20260807 15:15:42.831254 13777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:42.831549 13777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:42.831629 13777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:42.835855 13777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:42.835927 13777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:42.836009 13777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:42.840788 13777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:43855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:42.841943 13777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:42.843351 13777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:42.853374 13790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.853410 13787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.853374 13786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:42.853590 13777 server_base.cc:1034] running on GCE node
I20260807 15:15:42.854010 13777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:42.854643 13777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:42.855837 13777 hybrid_clock.cc:648] HybridClock initialized: now 1786115742855823 us; error 26 us; skew 500 ppm
I20260807 15:15:42.857758 13777 webserver.cc:460] Webserver started at http://127.31.230.195:45499/ using document root <none> and password file <none>
I20260807 15:15:42.858348 13777 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:42.858446 13777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:42.858688 13777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:42.860742 13777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "adfae2307a4a42f6846907e8e668e18d"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "d8b517aef98047e2ce43c76dac1c321f"
server_key_iv: "e590f751b4b823d67b6b4b4cadedbb2b"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.861330 13777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "adfae2307a4a42f6846907e8e668e18d"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "d8b517aef98047e2ce43c76dac1c321f"
server_key_iv: "e590f751b4b823d67b6b4b4cadedbb2b"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.864964 13777 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:15:42.867496 13799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.868709 13777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:42.868845 13777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "adfae2307a4a42f6846907e8e668e18d"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-gkw7"
server_key: "d8b517aef98047e2ce43c76dac1c321f"
server_key_iv: "e590f751b4b823d67b6b4b4cadedbb2b"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.868959 13777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:42.880192 13777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:42.880847 13777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:42.881043 13777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:42.881699 13777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:42.882642 13777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:42.882725 13777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.882799 13777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:42.882854 13777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.892331 13777 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:40659
I20260807 15:15:42.892351 13960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:40659 every 8 connection(s)
I20260807 15:15:42.893287 13777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:42.901398 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 13777
I20260807 15:15:42.901520 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:42.901849 32667 external_mini_cluster.cc:1428] Setting key f29f3d84d3aa6dc8e469ed4786361835
I20260807 15:15:42.903429 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.895294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:58787 (local address 127.31.230.254:43855)
0807 15:15:42.895456 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.895461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.896615 (+ 1154us) server_negotiation.cc:407] Connection header received
0807 15:15:42.898058 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.898064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.898128 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.898204 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.899207 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.899766 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.900986 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.901316 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.901438 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.902705 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:42.902710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:42.902713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:42.902726 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:42.902749 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:42.902760 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:42.902834 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:42.903231 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 15:15:42.903315 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:15:42.904400 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.895779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:43855 (local address 127.31.230.195:58787)
0807 15:15:42.896464 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:42.896506 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:42.897848 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:42.898364 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:42.898371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:42.898452 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:42.899041 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:42.899062 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.899911 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.899915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:42.900803 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:42.900813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.901460 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.901482 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:42.901702 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.902382 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:42.902384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:42.902424 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:42.902512 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:42.903065 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:42.903075 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:42.903082 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:15:42.903225 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 15:15:42.903518 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260807 15:15:42.905375 13961 heartbeater.cc:347] Connected to a master server at 127.31.230.254:43855
I20260807 15:15:42.905668 13961 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:42.906286 13961 heartbeater.cc:511] Master 127.31.230.254:43855 requested a full tablet report, sending...
I20260807 15:15:42.907258 13317 ts_manager.cc:194] Registered new tserver with Master: adfae2307a4a42f6846907e8e668e18d (127.31.230.195:40659)
I20260807 15:15:42.907790 13317 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.195:58787
I20260807 15:15:42.915963 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:15:42.918984 13362 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:42.917928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.31.230.254:43855)
0807 15:15:42.918099 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.918104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.918289 (+ 185us) server_negotiation.cc:407] Connection header received
0807 15:15:42.918451 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.918454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.918507 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.918589 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.918855 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41628: BlockingRecv error: recv got EOF from 127.0.0.1:41628 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:15:42.919114 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 13393
I20260807 15:15:42.937197 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 13575
I20260807 15:15:42.946269 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 13777
I20260807 15:15:42.958781 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 13271
2026-08-07T15:15:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (905 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T15:15:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:42Z Disabled control of system clock
I20260807 15:15:42.994091 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44433
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:38075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:44433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:15:43.111485 13972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:43.111745 13972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:43.111783 13972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:43.115618 13972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:43.115665 13972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:43.115681 13972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:43.115697 13972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:43.115720 13972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:43.120380 13972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:44433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:43.121390 13972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:43.122865 13972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:43.131469 13982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.131515 13981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.131479 13986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:43.131567 13972 server_base.cc:1034] running on GCE node
I20260807 15:15:43.132299 13972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:43.133385 13972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:43.134545 13972 hybrid_clock.cc:648] HybridClock initialized: now 1786115743134532 us; error 38 us; skew 500 ppm
I20260807 15:15:43.136564 13972 webserver.cc:460] Webserver started at http://127.31.230.254:38869/ using document root <none> and password file <none>
I20260807 15:15:43.137164 13972 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:43.137230 13972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:43.137513 13972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:43.139746 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "629a2af767ee4fb883abd7d3f22f47a4"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "ebf26927034551e6c763be49bc0447c7"
server_key_iv: "ae4eb8dfa714a5f0afa3699b2d9f1f3d"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.140324 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "629a2af767ee4fb883abd7d3f22f47a4"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "ebf26927034551e6c763be49bc0447c7"
server_key_iv: "ae4eb8dfa714a5f0afa3699b2d9f1f3d"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.144078 13972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 15:15:43.146538 13994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.147747 13972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:43.147879 13972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "629a2af767ee4fb883abd7d3f22f47a4"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "ebf26927034551e6c763be49bc0447c7"
server_key_iv: "ae4eb8dfa714a5f0afa3699b2d9f1f3d"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.147982 13972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:43.152608 13972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:43.153235 13972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:43.153421 13972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:43.160133 13972 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:44433
I20260807 15:15:43.160157 14057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:44433 every 8 connection(s)
I20260807 15:15:43.161103 13972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:43.164583 14058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:43.170207 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 13972
I20260807 15:15:43.170313 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:43.170390 14058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4: Bootstrap starting.
I20260807 15:15:43.170673 32667 external_mini_cluster.cc:1428] Setting key c1d8430d296f7bcced499463962e6ded
I20260807 15:15:43.173132 14058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:43.173952 14058 log.cc:824] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:43.176029 14058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4: No bootstrap required, opened a new log
I20260807 15:15:43.176625 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.172080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.31.230.254:44433)
0807 15:15:43.172940 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.172957 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.172995 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:15:43.173754 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.173774 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.174166 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.174505 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.175324 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:43.175329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:43.175341 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:43.175351 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:43.175406 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:43.175423 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:43.175540 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:43.175652 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:15:43.175896 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":140,"threads_started":1}
I20260807 15:15:43.179688 14058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629a2af767ee4fb883abd7d3f22f47a4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44433 } }
I20260807 15:15:43.179934 14058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:43.180013 14058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629a2af767ee4fb883abd7d3f22f47a4, State: Initialized, Role: FOLLOWER
I20260807 15:15:43.180451 14058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629a2af767ee4fb883abd7d3f22f47a4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44433 } }
I20260807 15:15:43.180605 14058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:43.180691 14058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:43.180811 14058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:43.181849 14058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629a2af767ee4fb883abd7d3f22f47a4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44433 } }
I20260807 15:15:43.182192 14058 leader_election.cc:302] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 629a2af767ee4fb883abd7d3f22f47a4; no voters:
I20260807 15:15:43.182550 14058 leader_election.cc:288] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:43.182746 14066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:43.183094 14066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [term 1 LEADER]: Becoming Leader. State: Replica: 629a2af767ee4fb883abd7d3f22f47a4, State: Running, Role: LEADER
I20260807 15:15:43.183449 14066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629a2af767ee4fb883abd7d3f22f47a4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44433 } }
I20260807 15:15:43.183928 14058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:43.185429 14072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629a2af767ee4fb883abd7d3f22f47a4. Latest consensus state: current_term: 1 leader_uuid: "629a2af767ee4fb883abd7d3f22f47a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629a2af767ee4fb883abd7d3f22f47a4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44433 } } }
I20260807 15:15:43.185361 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "629a2af767ee4fb883abd7d3f22f47a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629a2af767ee4fb883abd7d3f22f47a4" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44433 } } }
I20260807 15:15:43.185549 14072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:43.185570 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4 [sys.catalog]: This master's current role is: LEADER
W20260807 15:15:43.189442 14090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:43.189512 14090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:43.189657 14082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:43.190584 14082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:43.196621 14082 catalog_manager.cc:1347] Generated new cluster ID: 20e37aec8a6e40b2b649c764ffc2e134
I20260807 15:15:43.196703 14082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:43.208336 14082 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:43.208662 14082 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:43.214416 14082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 629a2af767ee4fb883abd7d3f22f47a4: Generated new TSK 0
I20260807 15:15:43.215108 14082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:43.217737 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44433
--builtin_ntp_servers=127.31.230.212:38075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:15:43.346553 14095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:43.346819 14095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:43.346858 14095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:43.350986 14095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:43.351042 14095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:43.351122 14095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:43.356338 14095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:43.357479 14095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:43.358927 14095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:43.368745 14106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.368811 14112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.368732 14108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:43.369114 14095 server_base.cc:1034] running on GCE node
I20260807 15:15:43.369614 14095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:43.370357 14095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:43.371534 14095 hybrid_clock.cc:648] HybridClock initialized: now 1786115743371490 us; error 61 us; skew 500 ppm
I20260807 15:15:43.373615 14095 webserver.cc:460] Webserver started at http://127.31.230.193:42473/ using document root <none> and password file <none>
I20260807 15:15:43.374248 14095 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:43.374315 14095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:43.374569 14095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:43.376819 14095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "953d9b06847f44019e7bcf5ef258d7ae"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "18a681df33924cdbb1b8d569ccc58490"
server_key_iv: "03e802ffda61418e1baf7161c7e02b06"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.377451 14095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "953d9b06847f44019e7bcf5ef258d7ae"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "18a681df33924cdbb1b8d569ccc58490"
server_key_iv: "03e802ffda61418e1baf7161c7e02b06"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.381507 14095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:43.384407 14119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.385854 14095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:15:43.385990 14095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "953d9b06847f44019e7bcf5ef258d7ae"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "18a681df33924cdbb1b8d569ccc58490"
server_key_iv: "03e802ffda61418e1baf7161c7e02b06"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.386113 14095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:43.390897 14095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:43.391539 14095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:43.391737 14095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:43.392321 14095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:43.393265 14095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:43.393340 14095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.393412 14095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:43.393453 14095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.402725 14095 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:43843
I20260807 15:15:43.402786 14276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:43843 every 8 connection(s)
I20260807 15:15:43.403733 14095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:43.406098 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 14095
I20260807 15:15:43.406193 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:43.406563 32667 external_mini_cluster.cc:1428] Setting key 328cabf519b866f19b92ff43e6efaeba
I20260807 15:15:43.408880 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44433
--builtin_ntp_servers=127.31.230.212:38075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 15:15:43.410678 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.405696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:41471 (local address 127.31.230.254:44433)
0807 15:15:43.405865 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.405871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.407177 (+ 1306us) server_negotiation.cc:407] Connection header received
0807 15:15:43.408844 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.408847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.408881 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.408947 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.410235 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:43.410238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:43.410240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:43.410259 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:43.410280 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:43.410287 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:43.410360 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:43.410536 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 15:15:43.410575 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:15:43.411938 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.406127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44433 (local address 127.31.230.193:41471)
0807 15:15:43.407021 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:43.407065 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:43.408605 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:43.409142 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:43.409154 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:43.409286 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:43.409915 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:43.409919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:43.409978 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:43.410074 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:43.410376 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:43.410388 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:43.410397 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:15:43.410581 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 15:15:43.410850 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":130,"threads_started":1}
I20260807 15:15:43.413183 14278 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44433
I20260807 15:15:43.413446 14278 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:43.413983 14278 heartbeater.cc:511] Master 127.31.230.254:44433 requested a full tablet report, sending...
I20260807 15:15:43.415784 14015 ts_manager.cc:194] Registered new tserver with Master: 953d9b06847f44019e7bcf5ef258d7ae (127.31.230.193:43843)
W20260807 15:15:43.532150 14284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:43.532490 14284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:43.532565 14284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:43.536877 14284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:43.536991 14284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:43.537107 14284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:43.542309 14284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:43.543521 14284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:43.545027 14284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:43.555871 14289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.555883 14293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.555871 14290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:43.555912 14284 server_base.cc:1034] running on GCE node
I20260807 15:15:43.556618 14284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:43.557320 14284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:43.558495 14284 hybrid_clock.cc:648] HybridClock initialized: now 1786115743558493 us; error 54 us; skew 500 ppm
I20260807 15:15:43.560411 14284 webserver.cc:460] Webserver started at http://127.31.230.194:45983/ using document root <none> and password file <none>
I20260807 15:15:43.561040 14284 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:43.561106 14284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:43.561381 14284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:43.563616 14284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "bbfaf000674b468782ce0fa3e748a3a5"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "020c89ce61bbb717013f55c444235f53"
server_key_iv: "3e58ea2eaa76469bcfb83befc100d931"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.564214 14284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "bbfaf000674b468782ce0fa3e748a3a5"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "020c89ce61bbb717013f55c444235f53"
server_key_iv: "3e58ea2eaa76469bcfb83befc100d931"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.568321 14284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:15:43.571169 14302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.572431 14284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:43.572588 14284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "bbfaf000674b468782ce0fa3e748a3a5"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "020c89ce61bbb717013f55c444235f53"
server_key_iv: "3e58ea2eaa76469bcfb83befc100d931"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.572716 14284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:43.577618 14284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:43.578269 14284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:43.578466 14284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:43.579038 14284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:43.580081 14284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:43.580157 14284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.580238 14284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:43.580289 14284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.590333 14284 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:35715
I20260807 15:15:43.590397 14460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:35715 every 8 connection(s)
I20260807 15:15:43.591347 14284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:43.597819 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 14284
I20260807 15:15:43.597978 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:43.598090 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.593444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:43549 (local address 127.31.230.254:44433)
0807 15:15:43.593657 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.593662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.594805 (+ 1143us) server_negotiation.cc:407] Connection header received
0807 15:15:43.596250 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.596254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.596302 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.596382 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.597555 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:43.597559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:43.597564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:43.597580 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:43.597604 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:43.597617 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:43.597730 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:43.597886 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 15:15:43.597938 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:15:43.598361 32667 external_mini_cluster.cc:1428] Setting key 2826a3e44b919d3d2b157fee6e097579
I20260807 15:15:43.599203 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.593911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44433 (local address 127.31.230.194:43549)
0807 15:15:43.594638 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:43.594681 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:43.596017 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:43.596554 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:43.596561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:43.596654 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:43.597208 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:43.597209 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:43.597271 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:43.597343 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:43.597710 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:43.597720 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:43.597726 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:15:43.597935 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 15:15:43.598197 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":124,"threads_started":1}
I20260807 15:15:43.600456 14461 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44433
I20260807 15:15:43.600705 14461 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:43.600693 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44433
--builtin_ntp_servers=127.31.230.212:38075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 15:15:43.601150 14461 heartbeater.cc:511] Master 127.31.230.254:44433 requested a full tablet report, sending...
I20260807 15:15:43.602345 14015 ts_manager.cc:194] Registered new tserver with Master: bbfaf000674b468782ce0fa3e748a3a5 (127.31.230.194:35715)
W20260807 15:15:43.720160 14464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:43.720440 14464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:43.720480 14464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:43.724537 14464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:43.724606 14464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:43.724665 14464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:43.729650 14464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:43.730734 14464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:43.732105 14464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:43.742126 14470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.742117 14473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.742117 14475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:43.742308 14464 server_base.cc:1034] running on GCE node
I20260807 15:15:43.742833 14464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:43.743481 14464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:43.744652 14464 hybrid_clock.cc:648] HybridClock initialized: now 1786115743744615 us; error 47 us; skew 500 ppm
I20260807 15:15:43.746603 14464 webserver.cc:460] Webserver started at http://127.31.230.195:40841/ using document root <none> and password file <none>
I20260807 15:15:43.747202 14464 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:43.747269 14464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:43.747534 14464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:43.749706 14464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "203e378824104d1bb837659cc49d0985"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "d87a8d3957810d3d1c27d22c3a173793"
server_key_iv: "bb2ec005b68466e9d301063ccf080c6f"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.750294 14464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "203e378824104d1bb837659cc49d0985"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "d87a8d3957810d3d1c27d22c3a173793"
server_key_iv: "bb2ec005b68466e9d301063ccf080c6f"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.754499 14464 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:15:43.757208 14483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.758522 14464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260807 15:15:43.758661 14464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "203e378824104d1bb837659cc49d0985"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-gkw7"
server_key: "d87a8d3957810d3d1c27d22c3a173793"
server_key_iv: "bb2ec005b68466e9d301063ccf080c6f"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.758791 14464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:43.763568 14464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:43.764204 14464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:43.764389 14464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:43.764950 14464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:43.765969 14464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:43.766098 14464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.766213 14464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:43.766283 14464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.775223 14464 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:37807
I20260807 15:15:43.775321 14646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:37807 every 8 connection(s)
I20260807 15:15:43.776257 14464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:43.778982 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 14464
I20260807 15:15:43.779104 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:43.779440 32667 external_mini_cluster.cc:1428] Setting key f250a7137dab2717360df806103d1db9
I20260807 15:15:43.783079 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.778361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:49549 (local address 127.31.230.254:44433)
0807 15:15:43.778508 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.778513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.779789 (+ 1276us) server_negotiation.cc:407] Connection header received
0807 15:15:43.781381 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.781386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.781432 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.781517 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.782728 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:43.782731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:43.782733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:43.782744 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:43.782765 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:43.782772 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:43.782845 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:43.782939 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 15:15:43.782977 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:15:43.784082 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.778727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44433 (local address 127.31.230.195:49549)
0807 15:15:43.779604 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:43.779660 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:43.781177 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:43.781740 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:43.781748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:43.781856 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:43.782482 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:43.782484 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:43.782526 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:43.782597 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:43.782831 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:43.782835 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:43.782839 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:43.782963 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 15:15:43.783209 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":118,"threads_started":1}
I20260807 15:15:43.785240 14647 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44433
I20260807 15:15:43.785583 14647 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:43.786031 14647 heartbeater.cc:511] Master 127.31.230.254:44433 requested a full tablet report, sending...
I20260807 15:15:43.787112 14015 ts_manager.cc:194] Registered new tserver with Master: 203e378824104d1bb837659cc49d0985 (127.31.230.195:37807)
I20260807 15:15:43.793507 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:15:43.796614 14061 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:43.795703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.31.230.254:44433)
0807 15:15:43.795825 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.795828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.796039 (+ 211us) server_negotiation.cc:407] Connection header received
0807 15:15:43.796146 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.796148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.796180 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.796265 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.796511 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58110: BlockingRecv error: recv got EOF from 127.0.0.1:58110 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:15:43.796862 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 14095
I20260807 15:15:43.815937 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 14284
I20260807 15:15:43.826226 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 14464
I20260807 15:15:43.837785 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 13972
2026-08-07T15:15:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (877 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:43 dist-test-slave-gkw7 krb5kdc[14663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:43 dist-test-slave-gkw7 krb5kdc[14663](info): set up 2 sockets
Aug 07 15:15:43 dist-test-slave-gkw7 krb5kdc[14663](info): commencing operation
krb5kdc: starting...
W20260807 15:15:45.078665 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:15:45.211877 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:40839
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:40839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:45.328215 14683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:45.328538 14683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:45.328620 14683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:45.332718 14683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:45.332815 14683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:45.332856 14683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:45.332916 14683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:45.332955 14683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:45.338358 14683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:40839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:40839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:45.339476 14683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:45.340539 14683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:45.349334 14692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:45.349334 14696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:45.349367 14691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:45.349677 14683 server_base.cc:1034] running on GCE node
I20260807 15:15:45.350338 14683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:45.351425 14683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:45.352615 14683 hybrid_clock.cc:648] HybridClock initialized: now 1786115745352576 us; error 52 us; skew 500 ppm
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:45.355427 14683 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:15:45.356321 14683 webserver.cc:460] Webserver started at http://127.31.230.254:33063/ using document root <none> and password file <none>
I20260807 15:15:45.356891 14683 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:45.356993 14683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:45.357234 14683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:45.359359 14683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "f6276a24ba1b4668a8d79b8f9e397c8b"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "296d51dea38aef7f9dcd9daea73586d4"
server_key_iv: "5b6961be4e45e4c72926feab50ef0502"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.359939 14683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "f6276a24ba1b4668a8d79b8f9e397c8b"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "296d51dea38aef7f9dcd9daea73586d4"
server_key_iv: "5b6961be4e45e4c72926feab50ef0502"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.363693 14683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 15:15:45.366122 14705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.367309 14683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:45.367455 14683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "f6276a24ba1b4668a8d79b8f9e397c8b"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "296d51dea38aef7f9dcd9daea73586d4"
server_key_iv: "5b6961be4e45e4c72926feab50ef0502"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.367568 14683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:45.386006 14683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:45.388934 14683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:45.389151 14683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:45.396731 14683 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:40839
I20260807 15:15:45.397025 14779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:40839 every 8 connection(s)
I20260807 15:15:45.397840 14683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:45.398857 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 14683
I20260807 15:15:45.398977 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:15:45.399284 32667 external_mini_cluster.cc:1428] Setting key 03477bf489a0c555b7e7b7848d1facfe
I20260807 15:15:45.401597 14780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:45.407274 14780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b: Bootstrap starting.
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:45.410086 14780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:45.410852 14780 log.cc:824] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:45.413146 14780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b: No bootstrap required, opened a new log
I20260807 15:15:45.413893 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:45.400772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.31.230.254:40839)
0807 15:15:45.401218 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:45.401233 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:45.401276 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:15:45.402027 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:45.402046 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:45.402388 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:45.402726 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:45.403571 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.404585 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.405471 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.406003 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.406184 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:45.409039 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:45.409044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:45.409057 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:45.409084 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:45.410869 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:45.411376 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:45.411382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:45.411389 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:45.411539 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:45.411880 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:45.411883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:45.411884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:45.412263 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:45.412404 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:45.412729 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:15:45.413047 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
I20260807 15:15:45.416714 14780 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40839 } }
I20260807 15:15:45.416922 14780 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:45.416998 14780 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6276a24ba1b4668a8d79b8f9e397c8b, State: Initialized, Role: FOLLOWER
I20260807 15:15:45.417449 14780 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40839 } }
I20260807 15:15:45.417647 14780 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:45.417717 14780 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:45.417834 14780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:45.418758 14780 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40839 } }
I20260807 15:15:45.419101 14780 leader_election.cc:302] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6276a24ba1b4668a8d79b8f9e397c8b; no voters:
I20260807 15:15:45.419454 14780 leader_election.cc:288] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:45.419551 14786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:45.419795 14786 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [term 1 LEADER]: Becoming Leader. State: Replica: f6276a24ba1b4668a8d79b8f9e397c8b, State: Running, Role: LEADER
I20260807 15:15:45.420147 14786 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40839 } }
I20260807 15:15:45.420792 14780 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:45.422163 14787 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40839 } } }
I20260807 15:15:45.422295 14787 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:45.422533 14788 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6276a24ba1b4668a8d79b8f9e397c8b. Latest consensus state: current_term: 1 leader_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6276a24ba1b4668a8d79b8f9e397c8b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40839 } } }
I20260807 15:15:45.422657 14788 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:45.423116 14799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:45.426025 14799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:45.431422 14799 catalog_manager.cc:1347] Generated new cluster ID: d324e32275af4bcb85f72b6434fd1abb
I20260807 15:15:45.431519 14799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:45.448882 14799 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:45.449880 14799 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:45.458701 14799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b: Generated new TSK 0
I20260807 15:15:45.459378 14799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:15:45.496289 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40839
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:45.612788 14818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:45.613088 14818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:45.613135 14818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:45.617259 14818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:45.617336 14818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:45.617446 14818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:15:45.623258 14818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:45.624383 14818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:45.625458 14818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:45.636096 14825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:45.636133 14828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:45.636197 14818 server_base.cc:1034] running on GCE node
W20260807 15:15:45.636487 14826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:45.636938 14818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:45.637686 14818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:45.638885 14818 hybrid_clock.cc:648] HybridClock initialized: now 1786115745638868 us; error 29 us; skew 500 ppm
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:45.641901 14818 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:15:45.642863 14818 webserver.cc:460] Webserver started at http://127.31.230.193:32977/ using document root <none> and password file <none>
I20260807 15:15:45.643535 14818 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:45.643607 14818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:45.643790 14818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:45.645944 14818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "9aea0f79165945ce98d8d0bdf5f000f3"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "9f1521736a6e3641f9677d90101b7270"
server_key_iv: "4e0bb55acece521ca4617374cf8912fa"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.646502 14818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "9aea0f79165945ce98d8d0bdf5f000f3"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "9f1521736a6e3641f9677d90101b7270"
server_key_iv: "4e0bb55acece521ca4617374cf8912fa"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.650391 14818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:15:45.652904 14836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.654268 14818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:45.654415 14818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "9aea0f79165945ce98d8d0bdf5f000f3"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "9f1521736a6e3641f9677d90101b7270"
server_key_iv: "4e0bb55acece521ca4617374cf8912fa"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.654531 14818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:45.691373 14818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:45.694702 14818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:45.694927 14818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:45.695582 14818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:45.696478 14818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:45.696529 14818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.696596 14818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:45.696631 14818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.705962 14818 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:37465
I20260807 15:15:45.705981 14998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:37465 every 8 connection(s)
I20260807 15:15:45.706950 14818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:45.714728 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 14818
I20260807 15:15:45.714881 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:45.715260 32667 external_mini_cluster.cc:1428] Setting key b53f0b5940441c6bd34d57ba3a31585a
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:45.722276 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:45.708874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:36071 (local address 127.31.230.254:40839)
0807 15:15:45.709025 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:45.709029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:45.709983 (+ 954us) server_negotiation.cc:407] Connection header received
0807 15:15:45.711460 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:45.711464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:45.711525 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:45.711636 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:45.712867 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.713429 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.714643 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.714930 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.715050 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:45.717920 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:45.717927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:45.717933 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:45.717969 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:45.719556 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:45.720254 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:45.720259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:45.720261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:45.720354 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:45.720787 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:45.720793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:45.720798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:45.721069 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:45.721184 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:45.721813 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 15:15:45.721979 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:15:45.723109 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:45.709286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40839 (local address 127.31.230.193:36071)
0807 15:15:45.709827 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:45.709872 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:45.711223 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:45.711833 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:45.711841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:45.712215 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:45.712711 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:45.712724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.713586 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.713592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:45.714505 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:45.714514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.715127 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.715133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:45.715366 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:45.716086 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:45.716108 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:45.717734 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:45.719680 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:45.719689 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:45.719709 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:45.720106 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:45.720452 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:45.720457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:45.720459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:45.720650 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:45.721187 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:45.721192 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:45.721445 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 15:15:45.721699 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 15:15:45.722080 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":121,"threads_started":1}
I20260807 15:15:45.724462 14999 heartbeater.cc:347] Connected to a master server at 127.31.230.254:40839
I20260807 15:15:45.724728 14999 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:45.725257 14999 heartbeater.cc:511] Master 127.31.230.254:40839 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:15:45.727130 14735 ts_manager.cc:194] Registered new tserver with Master: 9aea0f79165945ce98d8d0bdf5f000f3 (127.31.230.193:37465)
I20260807 15:15:45.728466 14735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:36071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:15:45.747854 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40839
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:45.865099 15005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:45.865378 15005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:45.865458 15005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:45.869481 15005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:45.869563 15005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:45.869650 15005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:15:45.874984 15005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:45.876057 15005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:45.877138 15005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:45.887306 15005 server_base.cc:1034] running on GCE node
W20260807 15:15:45.887422 15020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:45.887269 15014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:45.887321 15013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:45.888201 15005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:45.888973 15005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:45.890354 15005 hybrid_clock.cc:648] HybridClock initialized: now 1786115745890334 us; error 31 us; skew 500 ppm
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:45.893189 15005 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:15:45.894186 15005 webserver.cc:460] Webserver started at http://127.31.230.194:41009/ using document root <none> and password file <none>
I20260807 15:15:45.894764 15005 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:45.894856 15005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:45.895095 15005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:45.897158 15005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "6c34957c87e740c9bbdefb47453fabb5"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "b3c4a7225f545077e694981ae5b73051"
server_key_iv: "5062ccf3c8ccd6b592e134b108087cfd"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.897795 15005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "6c34957c87e740c9bbdefb47453fabb5"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "b3c4a7225f545077e694981ae5b73051"
server_key_iv: "5062ccf3c8ccd6b592e134b108087cfd"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.901643 15005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:15:45.904124 15032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.905332 15005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:45.905469 15005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "6c34957c87e740c9bbdefb47453fabb5"
format_stamp: "Formatted at 2026-08-07 15:15:45 on dist-test-slave-gkw7"
server_key: "b3c4a7225f545077e694981ae5b73051"
server_key_iv: "5062ccf3c8ccd6b592e134b108087cfd"
server_key_version: "encryptionkey@0"
I20260807 15:15:45.905635 15005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:45.923650 15005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:45.926755 15005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:45.926977 15005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:45.927603 15005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:45.928467 15005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:45.928539 15005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.928612 15005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:45.928665 15005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:45.937902 15005 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:41401
I20260807 15:15:45.937929 15189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:41401 every 8 connection(s)
I20260807 15:15:45.938887 15005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:45.945612 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 15005
I20260807 15:15:45.945734 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:45.946025 32667 external_mini_cluster.cc:1428] Setting key 99ee8d08757e7a5dccbeb230cf9d1a7b
Aug 07 15:15:45 dist-test-slave-gkw7 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:45.953418 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:45.940736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:50275 (local address 127.31.230.254:40839)
0807 15:15:45.940877 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:45.940887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:45.941765 (+ 878us) server_negotiation.cc:407] Connection header received
0807 15:15:45.943233 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:45.943236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:45.943295 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:45.943378 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:45.944599 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.945178 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.946580 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.946832 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.946952 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:45.949752 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:45.949756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:45.949758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:45.949783 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:45.951260 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:45.951814 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:45.951818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:45.951819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:45.951900 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:45.952275 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:45.952279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:45.952281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:45.952472 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:45.952583 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:45.953111 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 15:15:45.953245 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:15:45.954300 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:45.941086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40839 (local address 127.31.230.194:50275)
0807 15:15:45.941608 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:45.941653 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:45.942992 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:45.943551 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:45.943558 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:45.943946 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:45.944449 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:45.944462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.945332 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.945342 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:45.946420 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:45.946451 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:45.946986 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:45.946988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:45.947141 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:45.947817 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:45.947839 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:45.949515 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:45.951389 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:45.951395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:45.951413 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:45.951683 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:45.952015 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:45.952018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:45.952020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:45.952168 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:45.952583 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:45.952591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:45.952855 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 15:15:45.953048 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 15:15:45.953409 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":126,"threads_started":1}
I20260807 15:15:45.955530 15190 heartbeater.cc:347] Connected to a master server at 127.31.230.254:40839
I20260807 15:15:45.955808 15190 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:45.956471 15190 heartbeater.cc:511] Master 127.31.230.254:40839 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:15:45.957679 14735 ts_manager.cc:194] Registered new tserver with Master: 6c34957c87e740c9bbdefb47453fabb5 (127.31.230.194:41401)
I20260807 15:15:45.958526 14735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:50275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:15:45.974907 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40839
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:46.092980 15196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:46.093233 15196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:46.093276 15196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:46.097345 15196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:46.097409 15196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:46.097468 15196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:15:46.102820 15196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:15:46.103847 15196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:46.104844 15196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:46.114720 15201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:46.114742 15196 server_base.cc:1034] running on GCE node
W20260807 15:15:46.114773 15202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:46.114764 15205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:46.115367 15196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:46.115978 15196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:46.117141 15196 hybrid_clock.cc:648] HybridClock initialized: now 1786115746117124 us; error 30 us; skew 500 ppm
Aug 07 15:15:46 dist-test-slave-gkw7 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115746, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:46.119940 15196 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:15:46.120915 15196 webserver.cc:460] Webserver started at http://127.31.230.195:38337/ using document root <none> and password file <none>
I20260807 15:15:46.121642 15196 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:46.121744 15196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:46.122030 15196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:46.124166 15196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "a0e3d148d9e14ca8a8a4005010d59212"
format_stamp: "Formatted at 2026-08-07 15:15:46 on dist-test-slave-gkw7"
server_key: "d953986b718b7ddc679aa13f19f5ecd4"
server_key_iv: "1c044bccdc4caec96a12a47921aa0691"
server_key_version: "encryptionkey@0"
I20260807 15:15:46.124763 15196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "a0e3d148d9e14ca8a8a4005010d59212"
format_stamp: "Formatted at 2026-08-07 15:15:46 on dist-test-slave-gkw7"
server_key: "d953986b718b7ddc679aa13f19f5ecd4"
server_key_iv: "1c044bccdc4caec96a12a47921aa0691"
server_key_version: "encryptionkey@0"
I20260807 15:15:46.128631 15196 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:15:46.131188 15218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:46.132496 15196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:46.132643 15196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "a0e3d148d9e14ca8a8a4005010d59212"
format_stamp: "Formatted at 2026-08-07 15:15:46 on dist-test-slave-gkw7"
server_key: "d953986b718b7ddc679aa13f19f5ecd4"
server_key_iv: "1c044bccdc4caec96a12a47921aa0691"
server_key_version: "encryptionkey@0"
I20260807 15:15:46.132772 15196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:46.143429 15196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:46.146487 15196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:46.146708 15196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:46.147316 15196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:46.148159 15196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:46.148232 15196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:46.148305 15196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:46.148350 15196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:46.157599 15196 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:44555
I20260807 15:15:46.157627 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:44555 every 8 connection(s)
I20260807 15:15:46.158502 15196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:46.162171 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 15196
I20260807 15:15:46.162277 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:46.162595 32667 external_mini_cluster.cc:1428] Setting key f379b2415ba157f64db08b1533dfc6fe
Aug 07 15:15:46 dist-test-slave-gkw7 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115746, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:15:46.172695 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.160416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:47465 (local address 127.31.230.254:40839)
0807 15:15:46.160557 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.160560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.161435 (+ 875us) server_negotiation.cc:407] Connection header received
0807 15:15:46.163096 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.163099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.163160 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.163267 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:46.164444 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.165074 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.166334 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.166557 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.166679 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.169006 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:46.169015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:46.169017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:46.169041 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:46.170617 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.171142 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.171146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.171147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.171224 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.171583 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.171586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.171588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.171783 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:46.171895 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.172387 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.172514 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:15:46.173487 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.160804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40839 (local address 127.31.230.195:47465)
0807 15:15:46.161267 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.161326 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:46.162878 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:46.163446 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:46.163452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:46.163808 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:46.164302 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.164315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.165219 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.165222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.166207 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.166217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.166709 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.166714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.166888 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.167424 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:46.167440 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:46.168843 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:46.170739 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:46.170745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:46.170760 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:46.171020 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:46.171316 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:46.171320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:46.171322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:46.171484 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:46.171893 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:46.171900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:46.172142 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 15:15:46.172344 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 15:15:46.172666 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":135,"threads_started":1}
I20260807 15:15:46.174616 15379 heartbeater.cc:347] Connected to a master server at 127.31.230.254:40839
I20260807 15:15:46.174886 15379 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:46.175514 15379 heartbeater.cc:511] Master 127.31.230.254:40839 requested a full tablet report, sending...
I20260807 15:15:46.176579 14735 ts_manager.cc:194] Registered new tserver with Master: a0e3d148d9e14ca8a8a4005010d59212 (127.31.230.195:44555)
I20260807 15:15:46.177115 14735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:47465
I20260807 15:15:46.188264 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:46.198372 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.190397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.31.230.254:40839)
0807 15:15:46.190556 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.190561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.190732 (+ 171us) server_negotiation.cc:407] Connection header received
0807 15:15:46.190884 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.190887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.190928 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.191015 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:46.191762 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.192309 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.193172 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.193394 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.193508 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.195037 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:46.195041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:46.195044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:46.195070 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:46.196536 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.196975 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.196979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.196981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.197068 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.197365 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.197368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.197370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.197692 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:46.197835 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.198080 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.198198 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:15:46.201395 14735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:46.204442 14735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:46.219605 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.213939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:44555 (local address 127.0.0.1:37880)
0807 15:15:46.214177 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.214191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:46.214289 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:46.215195 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:46.215198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:46.215222 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:46.215393 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.215408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.216876 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.216879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.218193 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.218204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.219044 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.219048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.219153 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.219190 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 15:15:46.219307 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:15:46.219433 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":80,"thread_start_us":101,"threads_started":1}
I20260807 15:15:46.219611 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.213344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:37465 (local address 127.0.0.1:56748)
0807 15:15:46.213882 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.213895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:46.213963 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:46.215494 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:46.215497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:46.215515 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:46.215592 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.215597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.216670 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.216676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.218196 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.218204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.219219 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.219222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.219357 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.219402 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:15:46.219461 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:15:46.219517 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":100,"thread_start_us":99,"threads_started":1}
I20260807 15:15:46.219849 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.214466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.31.230.193:37465)
0807 15:15:46.215035 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.215052 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.215073 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:15:46.215143 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.215151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.215333 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.215711 (+ 378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:46.215836 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.216532 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.218324 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.219083 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.219433 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.219547 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.219593 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.219693 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":120,"threads_started":1}
I20260807 15:15:46.219942 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.213173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41401 (local address 127.0.0.1:42460)
0807 15:15:46.213714 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.213778 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:46.213915 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:46.215112 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:46.215121 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:46.215154 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:46.215373 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.215382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.216914 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.216918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.218060 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.218070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.219208 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.219212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.219360 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.219638 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 15:15:46.219707 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:15:46.219773 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":112,"thread_start_us":89,"threads_started":1}
I20260807 15:15:46.220449 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.213325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.31.230.194:41401)
0807 15:15:46.214563 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.214579 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.214606 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:15:46.214690 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.214695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.214935 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.215106 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:46.215518 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.216729 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.218185 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.219053 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.219287 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.219423 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.220191 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.220304 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130,"thread_start_us":175,"threads_started":1}
I20260807 15:15:46.220821 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.214313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.31.230.195:44555)
0807 15:15:46.214671 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.214691 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.214717 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:15:46.214795 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.214801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.215020 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.215349 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:46.215509 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.216727 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.218354 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.218908 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.219053 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.219192 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.220549 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.220659 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":169,"threads_started":1}
I20260807 15:15:46.223167 15100 tablet_service.cc:1467] Processing CreateTablet for tablet 47fb85616b1a40709da62cf847902b20 (DEFAULT_TABLE table=test-table [id=5384f60456ca4565844cbc4563bdde94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:46.223249 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 47fb85616b1a40709da62cf847902b20 (DEFAULT_TABLE table=test-table [id=5384f60456ca4565844cbc4563bdde94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:46.223436 15283 tablet_service.cc:1467] Processing CreateTablet for tablet 47fb85616b1a40709da62cf847902b20 (DEFAULT_TABLE table=test-table [id=5384f60456ca4565844cbc4563bdde94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:46.224268 15100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47fb85616b1a40709da62cf847902b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:46.224323 15283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47fb85616b1a40709da62cf847902b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:46.224319 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47fb85616b1a40709da62cf847902b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:46.230373 15406 tablet_bootstrap.cc:492] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Bootstrap starting.
I20260807 15:15:46.231802 15408 tablet_bootstrap.cc:492] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3: Bootstrap starting.
I20260807 15:15:46.233875 15409 tablet_bootstrap.cc:492] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Bootstrap starting.
I20260807 15:15:46.234129 15408 tablet_bootstrap.cc:654] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:46.234606 15406 tablet_bootstrap.cc:654] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:46.235123 15408 log.cc:824] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:46.235432 15406 log.cc:824] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:46.235926 15409 tablet_bootstrap.cc:654] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:46.237207 15409 log.cc:824] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:46.237493 15408 tablet_bootstrap.cc:492] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3: No bootstrap required, opened a new log
I20260807 15:15:46.237753 15408 ts_tablet_manager.cc:1397] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260807 15:15:46.238631 15406 tablet_bootstrap.cc:492] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: No bootstrap required, opened a new log
I20260807 15:15:46.238842 15406 ts_tablet_manager.cc:1397] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 15:15:46.239372 15409 tablet_bootstrap.cc:492] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: No bootstrap required, opened a new log
I20260807 15:15:46.239549 15409 ts_tablet_manager.cc:1397] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:15:46.241402 15408 raft_consensus.cc:348] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.241729 15408 raft_consensus.cc:374] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:46.241837 15408 raft_consensus.cc:729] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea0f79165945ce98d8d0bdf5f000f3, State: Initialized, Role: FOLLOWER
I20260807 15:15:46.242328 15408 consensus_queue.cc:260] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.242816 15406 raft_consensus.cc:348] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.243037 14999 heartbeater.cc:503] Master 127.31.230.254:40839 was elected leader, sending a full tablet report...
I20260807 15:15:46.243124 15406 raft_consensus.cc:374] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:46.243222 15406 raft_consensus.cc:729] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c34957c87e740c9bbdefb47453fabb5, State: Initialized, Role: FOLLOWER
I20260807 15:15:46.243427 15408 ts_tablet_manager.cc:1428] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:15:46.243356 15409 raft_consensus.cc:348] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.243620 15409 raft_consensus.cc:374] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:46.243726 15409 raft_consensus.cc:729] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e3d148d9e14ca8a8a4005010d59212, State: Initialized, Role: FOLLOWER
I20260807 15:15:46.244244 15406 consensus_queue.cc:260] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.244225 15409 consensus_queue.cc:260] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.245209 15190 heartbeater.cc:503] Master 127.31.230.254:40839 was elected leader, sending a full tablet report...
I20260807 15:15:46.245239 15409 ts_tablet_manager.cc:1428] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:15:46.245246 15406 ts_tablet_manager.cc:1428] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:15:46.245724 15379 heartbeater.cc:503] Master 127.31.230.254:40839 was elected leader, sending a full tablet report...
I20260807 15:15:46.342806 15416 raft_consensus.cc:482] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:46.343070 15416 raft_consensus.cc:504] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.344892 15416 leader_election.cc:288] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c34957c87e740c9bbdefb47453fabb5 (127.31.230.194:41401), 9aea0f79165945ce98d8d0bdf5f000f3 (127.31.230.193:37465)
I20260807 15:15:46.348230 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.344721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:40201 (local address 127.31.230.194:41401)
0807 15:15:46.344891 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.344896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.344917 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:15:46.345091 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.345094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.345175 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.345268 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:46.345618 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.346157 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.347230 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.347636 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.347776 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.347831 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.348028 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.348102 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:15:46.348224 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.344613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41401 (local address 127.31.230.195:40201)
0807 15:15:46.344806 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.344829 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:46.344955 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:46.345308 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:46.345314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:46.345339 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:46.345474 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.345481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.346297 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.346299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.347080 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.347089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.347774 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.347778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.347886 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.347909 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:15:46.348009 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 15:15:46.348086 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 15:15:46.348913 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.345120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:37465 (local address 127.31.230.195:60907)
0807 15:15:46.345648 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.345661 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:46.345769 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:46.346020 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:46.346023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:46.346039 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:46.346156 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.346161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.346896 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.346902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.347751 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:46.347761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.348490 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.348494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:46.348585 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.348614 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:15:46.348720 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:15:46.348787 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":185,"threads_started":1}
I20260807 15:15:46.348928 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.345327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:60907 (local address 127.31.230.193:37465)
0807 15:15:46.345458 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.345464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.345750 (+ 286us) server_negotiation.cc:407] Connection header received
0807 15:15:46.345875 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.345884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.345925 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.346009 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:46.346281 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.346776 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.347882 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.348325 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.348458 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.348501 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.348745 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.348819 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:15:46.348932 15128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47fb85616b1a40709da62cf847902b20" candidate_uuid: "a0e3d148d9e14ca8a8a4005010d59212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c34957c87e740c9bbdefb47453fabb5" is_pre_election: true
I20260807 15:15:46.349324 15128 raft_consensus.cc:2454] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e3d148d9e14ca8a8a4005010d59212 in term 0.
I20260807 15:15:46.349661 14933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47fb85616b1a40709da62cf847902b20" candidate_uuid: "a0e3d148d9e14ca8a8a4005010d59212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" is_pre_election: true
I20260807 15:15:46.349881 15219 leader_election.cc:302] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c34957c87e740c9bbdefb47453fabb5, a0e3d148d9e14ca8a8a4005010d59212; no voters:
I20260807 15:15:46.349956 14933 raft_consensus.cc:2454] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e3d148d9e14ca8a8a4005010d59212 in term 0.
I20260807 15:15:46.350181 15416 raft_consensus.cc:2781] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:46.350301 15416 raft_consensus.cc:482] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:46.350366 15416 raft_consensus.cc:3037] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:46.351454 15416 raft_consensus.cc:504] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.351820 15416 leader_election.cc:288] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [CANDIDATE]: Term 1 election: Requested vote from peers 6c34957c87e740c9bbdefb47453fabb5 (127.31.230.194:41401), 9aea0f79165945ce98d8d0bdf5f000f3 (127.31.230.193:37465)
I20260807 15:15:46.352243 15128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47fb85616b1a40709da62cf847902b20" candidate_uuid: "a0e3d148d9e14ca8a8a4005010d59212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c34957c87e740c9bbdefb47453fabb5"
I20260807 15:15:46.352262 14933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47fb85616b1a40709da62cf847902b20" candidate_uuid: "a0e3d148d9e14ca8a8a4005010d59212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea0f79165945ce98d8d0bdf5f000f3"
I20260807 15:15:46.352388 15128 raft_consensus.cc:3037] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:46.352397 14933 raft_consensus.cc:3037] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:46.353463 14933 raft_consensus.cc:2454] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e3d148d9e14ca8a8a4005010d59212 in term 1.
I20260807 15:15:46.353477 15128 raft_consensus.cc:2454] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e3d148d9e14ca8a8a4005010d59212 in term 1.
I20260807 15:15:46.353819 15222 leader_election.cc:302] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea0f79165945ce98d8d0bdf5f000f3, a0e3d148d9e14ca8a8a4005010d59212; no voters:
I20260807 15:15:46.354013 15416 raft_consensus.cc:2781] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:46.354260 15416 raft_consensus.cc:686] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 1 LEADER]: Becoming Leader. State: Replica: a0e3d148d9e14ca8a8a4005010d59212, State: Running, Role: LEADER
I20260807 15:15:46.354619 15416 consensus_queue.cc:237] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } }
I20260807 15:15:46.358058 14734 catalog_manager.cc:5495] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e3d148d9e14ca8a8a4005010d59212 (127.31.230.195). New cstate: current_term: 1 leader_uuid: "a0e3d148d9e14ca8a8a4005010d59212" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e3d148d9e14ca8a8a4005010d59212" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260807 15:15:46.374310 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.370721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.31.230.195:44555)
0807 15:15:46.370887 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.370890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.370904 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:15:46.371041 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.371044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.371102 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.371193 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:46.371438 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.371987 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.372870 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.373078 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.373169 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.373447 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:46.373927 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:46.374021 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.374081 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.374132 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:15:46.380810 14933 raft_consensus.cc:1261] T 47fb85616b1a40709da62cf847902b20 P 9aea0f79165945ce98d8d0bdf5f000f3 [term 1 FOLLOWER]: Refusing update from remote peer a0e3d148d9e14ca8a8a4005010d59212: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:46.380810 15128 raft_consensus.cc:1261] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 1 FOLLOWER]: Refusing update from remote peer a0e3d148d9e14ca8a8a4005010d59212: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:46.381719 15416 consensus_queue.cc:1035] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c34957c87e740c9bbdefb47453fabb5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:46.382001 15418 consensus_queue.cc:1035] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea0f79165945ce98d8d0bdf5f000f3" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:46.390013 15427 mvcc.cc:205] Tried to move back new op lower bound from 7315930097167077376 to 7315930097071579136. Current Snapshot: MvccSnapshot[applied={T|T < 7315930097167077376}]
I20260807 15:15:46.391223 15425 mvcc.cc:205] Tried to move back new op lower bound from 7315930097167077376 to 7315930097071579136. Current Snapshot: MvccSnapshot[applied={T|T < 7315930097167077376}]
I20260807 15:15:46.399930 14735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:46.400102 14735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:46.402755 14735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6276a24ba1b4668a8d79b8f9e397c8b: Sending DeleteTablet for 3 replicas of tablet 47fb85616b1a40709da62cf847902b20
I20260807 15:15:46.403513 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 47fb85616b1a40709da62cf847902b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:46 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:56748
I20260807 15:15:46.403523 15100 tablet_service.cc:1514] Processing DeleteTablet for tablet 47fb85616b1a40709da62cf847902b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:46 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:42460
I20260807 15:15:46.404016 15439 tablet_replica.cc:330] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: stopping tablet replica
I20260807 15:15:46.404354 15439 raft_consensus.cc:2229] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:46.404433 15283 tablet_service.cc:1514] Processing DeleteTablet for tablet 47fb85616b1a40709da62cf847902b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:46 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:37880
I20260807 15:15:46.404681 15439 raft_consensus.cc:2258] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:46.405042 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 14818
I20260807 15:15:46.406215 15439 ts_tablet_manager.cc:1905] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:46.408628 15439 ts_tablet_manager.cc:1918] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:46.408720 15439 log.cc:1196] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/47fb85616b1a40709da62cf847902b20
I20260807 15:15:46.409008 15439 ts_tablet_manager.cc:1939] T 47fb85616b1a40709da62cf847902b20 P 6c34957c87e740c9bbdefb47453fabb5: Deleting consensus metadata
I20260807 15:15:46.410285 14707 catalog_manager.cc:4841] TS 6c34957c87e740c9bbdefb47453fabb5 (127.31.230.194:41401): tablet 47fb85616b1a40709da62cf847902b20 (table test-table [id=5384f60456ca4565844cbc4563bdde94]) successfully deleted
I20260807 15:15:46.419960 15442 tablet_replica.cc:330] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: stopping tablet replica
I20260807 15:15:46.420334 15442 raft_consensus.cc:2229] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:46.420750 15442 raft_consensus.cc:2258] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:46.422523 15442 ts_tablet_manager.cc:1905] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:46.425035 15442 ts_tablet_manager.cc:1918] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:46.425169 15442 log.cc:1196] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/47fb85616b1a40709da62cf847902b20
I20260807 15:15:46.425491 15442 ts_tablet_manager.cc:1939] T 47fb85616b1a40709da62cf847902b20 P a0e3d148d9e14ca8a8a4005010d59212: Deleting consensus metadata
W20260807 15:15:46.426213 14710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:37465 (error 108)
I20260807 15:15:46.426383 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 15005
I20260807 15:15:46.432343 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.431901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:37465 (local address 127.0.0.1:56752)
0807 15:15:46.432088 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:46.432212 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:37465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:15:46.432600 14710 catalog_manager.cc:4584] TS 9aea0f79165945ce98d8d0bdf5f000f3 (127.31.230.193:37465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47fb85616b1a40709da62cf847902b20: Network error: Client connection negotiation failed: client connection to 127.31.230.193:37465: connect: Connection refused (error 111)
I20260807 15:15:46.432900 14710 catalog_manager.cc:4841] TS a0e3d148d9e14ca8a8a4005010d59212 (127.31.230.195:44555): tablet 47fb85616b1a40709da62cf847902b20 (table test-table [id=5384f60456ca4565844cbc4563bdde94]) successfully deleted
I20260807 15:15:46.440707 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 15196
I20260807 15:15:46.458746 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 14683
2026-08-07T15:15:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2626 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:46 dist-test-slave-gkw7 krb5kdc[15446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:46 dist-test-slave-gkw7 krb5kdc[15446](info): set up 2 sockets
Aug 07 15:15:46 dist-test-slave-gkw7 krb5kdc[15446](info): commencing operation
krb5kdc: starting...
W20260807 15:15:47.684352 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:15:47 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/rangeradmin_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/rangeradmin_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/rangerlookup_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/rangerlookup_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
I20260807 15:15:47.881377 32667 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLoCTfz/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/postgres -l logfile start
2026-08-07 15:15:50.577 UTC [15487] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:15:50.577 UTC [15487] LOG: listening on IPv4 address "127.31.230.212", port 51919
2026-08-07 15:15:50.581 UTC [15487] LOG: listening on Unix socket "/tmp/.s.PGSQL.51919"
2026-08-07 15:15:50.604 UTC [15491] LOG: database system was shut down at 2026-08-07 15:15:49 UTC
2026-08-07 15:15:50.608 UTC [15487] LOG: database system is ready to accept connections
I20260807 15:15:51.729648 32667 mini_postgres.cc:96] Postgres bound to 51919
2026-08-07 15:15:51.738 UTC [15501] FATAL: database "slave" does not exist
127.31.230.212:51919 - accepting connections
I20260807 15:15:51.739078 32667 mini_ranger.cc:161] Starting Ranger...
I20260807 15:15:51.756356 32667 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:15:52.302346 32667 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:15:52.302443 32667 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-admin
2026-08-07 15:15:52,641 [I] DB FLAVOR :POSTGRES
2026-08-07 15:15:52,641 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:15:52,641 [I] Checking connection..
2026-08-07 15:15:52,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:15:52,913 [I] Checking connection passed.
2026-08-07 15:15:52,913 [I] --------- Verifying version history table ---------
2026-08-07 15:15:52,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:15:53,185 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:15:53,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:15:53,452 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:15:53,452 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:15:53,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:15:53.720 UTC [15578] WARNING: there is no transaction in progress
2026-08-07 15:15:53,729 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:15:53,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:15:53,970 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:15:53,970 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:15:53,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:15:54,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:15:54,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-07 15:15:54,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:15:54,984 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:15:54,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:15:55,252 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:15:55,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:15:55,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:15:55,770 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:15:55,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:15:56.797 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.814 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.831 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.848 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.864 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.887 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.945 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.952 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.967 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.977 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:56.988 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:57.003 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:57.013 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:57.020 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:57.027 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:57.034 UTC [15721] WARNING: there is no transaction in progress
2026-08-07 15:15:57,739 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:15:57,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:15:58,012 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:15:58,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:15:58,257 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:15:58,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:15:58,529 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:15:58,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:15:58,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:15:59,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-07 15:15:59,316 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:15:59,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:15:59,589 [I] DB_PATCHES have already been applied
I20260807 15:15:59.595474 32667 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLoCTfz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:15:59.595556 32667 mini_ranger.cc:193] Using host: 127.31.230.212
I20260807 15:15:59.600452 32667 mini_ranger.cc:227] Ranger admin URL: http://127.31.230.212:48047
Aug 07, 2026 3:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:16:01.055634 32667 mini_ranger.cc:160] Time spent starting Ranger: real 9.317s user 0.000s sys 0.014s
Aug 07, 2026 3:16:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:16:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 3:16:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:16:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:16:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:16:23.583217 32667 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/rangerkms_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/rangerkms_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.230.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.230.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:16:23 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115783, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:16:23.683224 32667 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:16:23.694790 32667 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:16:24.668607 32667 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:16:24.668692 32667 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms
I20260807 15:16:24.668794 32667 mini_ranger_kms.cc:227] Using postgres at 127.31.230.212:51919
2026-08-07 15:16:25,227 [I] DB FLAVOR :POSTGRES
2026-08-07 15:16:25,228 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:16:25,228 [I] Checking connection
2026-08-07 15:16:25,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:16:25,493 [I] connection success
2026-08-07 15:16:25,494 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:16:25,494 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:16:25,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:16:25,758 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:16:25,759 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:16:25,759 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:16:25,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:51919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:16:26,025 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:16:26.550550 32667 mini_ranger_kms.cc:310] Created kms service
I20260807 15:16:26.773685 32667 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:16:26.821858 32667 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:16:26 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115783, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:16:27.250609 32667 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:16:27.250787 32667 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:16:27.250818 32667 mini_ranger_kms.cc:240] Using host: 127.31.230.212
I20260807 15:16:27.255820 32667 mini_ranger_kms.cc:276] Ranger KMS PID: 15975
I20260807 15:16:27.255985 32667 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.230.212:33107
Aug 07, 2026 3:16:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:16:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:16:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:16:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:16:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:16:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:16:29 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115789, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.230.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:29.948714 32667 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.265s user 0.023s sys 0.229s
I20260807 15:16:29.948855 32667 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:16:29.948963 32667 mini_ranger_kms.cc:401] 127.31.230.212:33107/kms/v1/keys
Aug 07 15:16:30 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115789, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.230.212@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
Aug 07 15:16:30 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115789, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.230.212@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
Aug 07 15:16:32 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:16:32.857161 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:41425
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:41425
--ranger_config_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:33107/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:33.025709 16042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:33.026036 16042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:33.026122 16042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:33.030447 16042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:33.030544 16042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:33.030598 16042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:33.030659 16042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:33.030695 16042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:33.036311 16042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.230.212:33107/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:41425
--ranger_config_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:41425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:16:33.037464 16042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:33.038666 16042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:33.049044 16047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:33.049084 16048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:33.049090 16050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:33.050092 16042 server_base.cc:1034] running on GCE node
I20260807 15:16:33.050864 16042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:33.052424 16042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:33.053647 16042 hybrid_clock.cc:648] HybridClock initialized: now 1786115793053609 us; error 40 us; skew 500 ppm
Aug 07 15:16:33 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:33.057307 16042 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:16:33.058406 16042 webserver.cc:460] Webserver started at http://127.31.230.254:35075/ using document root <none> and password file <none>
I20260807 15:16:33.059006 16042 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:33.059090 16042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:33.059324 16042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:16:33 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:16:33.112314 16042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "b107eadb1bf847dc80f3a28ce8918fd8"
format_stamp: "Formatted at 2026-08-07 15:16:33 on dist-test-slave-gkw7"
server_key: "09576f2265ba8461fa65e7d67b498f82"
server_key_iv: "b0fd48aeae0752c8f27e1a41bf3d0950"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:33.113086 16042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "b107eadb1bf847dc80f3a28ce8918fd8"
format_stamp: "Formatted at 2026-08-07 15:16:33 on dist-test-slave-gkw7"
server_key: "09576f2265ba8461fa65e7d67b498f82"
server_key_iv: "b0fd48aeae0752c8f27e1a41bf3d0950"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:33.118367 16042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 15:16:33.133996 16058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:33.135582 16042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:16:33.135761 16042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "b107eadb1bf847dc80f3a28ce8918fd8"
format_stamp: "Formatted at 2026-08-07 15:16:33 on dist-test-slave-gkw7"
server_key: "09576f2265ba8461fa65e7d67b498f82"
server_key_iv: "b0fd48aeae0752c8f27e1a41bf3d0950"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:33.135919 16042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:33.168725 16042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:33.172238 16042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:33.172487 16042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:33.181295 16042 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:41425
I20260807 15:16:33.181339 16124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:41425 every 8 connection(s)
I20260807 15:16:33.182535 16042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:16:33.185451 16126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:33.188321 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 16042
I20260807 15:16:33.188458 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:16:33.188828 32667 external_mini_cluster.cc:1428] Setting key 237d45084f90ae4bd04fcdfc5163a5a8
I20260807 15:16:33.192092 16126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8: Bootstrap starting.
I20260807 15:16:33.194772 16126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:33.195719 16126 log.cc:824] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:33.198079 16126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8: No bootstrap required, opened a new log
Aug 07 15:16:33 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:16:33.202535 16126 raft_consensus.cc:348] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41425 } }
I20260807 15:16:33.202812 16126 raft_consensus.cc:374] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:16:33.202927 16126 raft_consensus.cc:729] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b107eadb1bf847dc80f3a28ce8918fd8, State: Initialized, Role: FOLLOWER
I20260807 15:16:33.203466 16126 consensus_queue.cc:260] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41425 } }
I20260807 15:16:33.203701 16126 raft_consensus.cc:388] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:16:33.203794 16126 raft_consensus.cc:482] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:16:33.203933 16126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:33.205117 16126 raft_consensus.cc:504] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41425 } }
I20260807 15:16:33.205475 16126 leader_election.cc:302] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b107eadb1bf847dc80f3a28ce8918fd8; no voters:
I20260807 15:16:33.205906 16126 leader_election.cc:288] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:16:33.206027 16144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:16:33.206193 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:33.190391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.31.230.254:41425)
0807 15:16:33.190900 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:33.190919 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:33.190964 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:16:33.191800 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:33.191825 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:33.192257 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:33.192629 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:33.193594 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:33.195135 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:33.196341 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:33.197000 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:33.197172 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:33.200389 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:33.200396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:33.200413 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:33.200444 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:33.202646 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:33.203178 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:33.203184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:33.203191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:33.203328 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:33.203728 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:33.203731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:33.203733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:33.204240 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:33.204440 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:33.204738 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:16:33.205181 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":152,"threads_started":1}
I20260807 15:16:33.206272 16144 raft_consensus.cc:686] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [term 1 LEADER]: Becoming Leader. State: Replica: b107eadb1bf847dc80f3a28ce8918fd8, State: Running, Role: LEADER
I20260807 15:16:33.207023 16144 consensus_queue.cc:237] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41425 } }
I20260807 15:16:33.207298 16126 sys_catalog.cc:564] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:33.210541 16146 sys_catalog.cc:455] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b107eadb1bf847dc80f3a28ce8918fd8. Latest consensus state: current_term: 1 leader_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41425 } } }
I20260807 15:16:33.210624 16126 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:16:33.210669 16146 sys_catalog.cc:458] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:33.210983 16145 sys_catalog.cc:455] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b107eadb1bf847dc80f3a28ce8918fd8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41425 } } }
I20260807 15:16:33.211057 16145 sys_catalog.cc:458] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:33.212365 16162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:16:33.215745 16162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:16:33.229420 16162 catalog_manager.cc:1347] Generated new cluster ID: 67bdd0f17b54409399712978efa2a432
I20260807 15:16:33.229498 16162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:16:33.250680 16162 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:16:33.252082 16162 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:16:33.258490 16162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8: Generated new TSK 0
I20260807 15:16:33.259431 16162 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:16:34.060958 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:33107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:41425
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:34.237993 16200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:34.238406 16200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:34.238480 16200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:34.244370 16200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:34.244451 16200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:34.244560 16200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:16:34.251502 16200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.230.212:33107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:41425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:16:34.252790 16200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:34.254143 16200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:34.267872 16212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:34.268404 16210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:34.269743 16200 server_base.cc:1034] running on GCE node
W20260807 15:16:34.270330 16209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:34.270952 16200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:34.271816 16200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:34.273128 16200 hybrid_clock.cc:648] HybridClock initialized: now 1786115794273058 us; error 88 us; skew 500 ppm
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:34.277223 16200 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:16:34.278427 16200 webserver.cc:460] Webserver started at http://127.31.230.193:43203/ using document root <none> and password file <none>
I20260807 15:16:34.279227 16200 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:34.279367 16200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:34.279691 16200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:16:34.297250 16200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "b80ceca5d75245d59411c04149cde4b6"
format_stamp: "Formatted at 2026-08-07 15:16:34 on dist-test-slave-gkw7"
server_key: "bb2b1d8bce4142fab0a511621c170708"
server_key_iv: "ef1a94471014eb14907dedf475162b24"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:34.298180 16200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "b80ceca5d75245d59411c04149cde4b6"
format_stamp: "Formatted at 2026-08-07 15:16:34 on dist-test-slave-gkw7"
server_key: "bb2b1d8bce4142fab0a511621c170708"
server_key_iv: "ef1a94471014eb14907dedf475162b24"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:34.304656 16200 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 15:16:34.332314 16222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:34.335763 16200 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260807 15:16:34.335948 16200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "b80ceca5d75245d59411c04149cde4b6"
format_stamp: "Formatted at 2026-08-07 15:16:34 on dist-test-slave-gkw7"
server_key: "bb2b1d8bce4142fab0a511621c170708"
server_key_iv: "ef1a94471014eb14907dedf475162b24"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:34.336129 16200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:34.364559 16200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:34.368656 16200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:34.368871 16200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:34.369686 16200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:16:34.370800 16200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:16:34.370868 16200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:34.370927 16200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:16:34.371006 16200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 15:16:34.383592 16200 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:43891
I20260807 15:16:34.384868 16200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:16:34.392247 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 16200
I20260807 15:16:34.392524 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:16:34.393359 32667 external_mini_cluster.cc:1428] Setting key 910137a1e46b68d09a8f3b48363d2d22
I20260807 15:16:34.394379 16379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:43891 every 8 connection(s)
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:16:34.416775 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:34.389836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:39059 (local address 127.31.230.254:41425)
0807 15:16:34.390314 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:34.390321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:34.391711 (+ 1390us) server_negotiation.cc:407] Connection header received
0807 15:16:34.399346 (+ 7635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:34.399351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:34.399421 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:34.399524 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:34.401376 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.402605 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.405939 (+ 3334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.406557 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.406728 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:34.410921 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:34.410928 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:34.410934 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:34.410973 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:34.413210 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:34.414059 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:34.414064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:34.414067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:34.414186 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:34.414992 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:34.414996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:34.414998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:34.415302 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:34.415413 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:34.416287 (+ 874us) server_negotiation.cc:299] Negotiation successful
0807 15:16:34.416455 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":97,"threads_started":1}
I20260807 15:16:34.419204 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:34.388596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:41425 (local address 127.31.230.193:39059)
0807 15:16:34.391507 (+ 2911us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:34.391588 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:34.392780 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:34.399930 (+ 7150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:34.399943 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:34.400372 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:34.401142 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:34.401167 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.402976 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.402983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:34.405669 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:34.405688 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.407168 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.407176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:34.407474 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:34.408419 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:34.408447 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:34.410683 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:34.413398 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:34.413407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:34.413432 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:34.413905 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:34.414339 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:34.414344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:34.414349 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:34.414841 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:34.415508 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:34.415517 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:34.415936 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 15:16:34.417384 (+ 1448us) client_negotiation.cc:240] Negotiation successful
0807 15:16:34.417856 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2655,"thread_start_us":161,"threads_started":1}
I20260807 15:16:34.421277 16380 heartbeater.cc:347] Connected to a master server at 127.31.230.254:41425
I20260807 15:16:34.421608 16380 heartbeater.cc:464] Registering TS with master...
I20260807 15:16:34.422323 16380 heartbeater.cc:511] Master 127.31.230.254:41425 requested a full tablet report, sending...
I20260807 15:16:34.424636 16078 ts_manager.cc:194] Registered new tserver with Master: b80ceca5d75245d59411c04149cde4b6 (127.31.230.193:43891)
I20260807 15:16:34.427186 16078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:39059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:16:34.456887 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:33107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:41425
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:34.598675 16388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:34.599038 16388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:34.599156 16388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:34.604050 16388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:34.604143 16388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:34.604259 16388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:16:34.612192 16388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.230.212:33107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:41425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:16:34.613677 16388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:34.615314 16388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:34.629853 16400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:34.629897 16398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:34.630323 16388 server_base.cc:1034] running on GCE node
W20260807 15:16:34.630519 16397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:34.631112 16388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:34.632040 16388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:34.633322 16388 hybrid_clock.cc:648] HybridClock initialized: now 1786115794633299 us; error 42 us; skew 500 ppm
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:34.637422 16388 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:16:34.638756 16388 webserver.cc:460] Webserver started at http://127.31.230.194:34281/ using document root <none> and password file <none>
I20260807 15:16:34.639647 16388 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:34.639761 16388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:34.640123 16388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:16:34.664825 16388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "cc35e1fbcc954545a7320603ea1fdf28"
format_stamp: "Formatted at 2026-08-07 15:16:34 on dist-test-slave-gkw7"
server_key: "9c11e021e6cd85fe7bb31b353310ee7d"
server_key_iv: "08119cfec5d88c807e571745d86b55d2"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:34.665877 16388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "cc35e1fbcc954545a7320603ea1fdf28"
format_stamp: "Formatted at 2026-08-07 15:16:34 on dist-test-slave-gkw7"
server_key: "9c11e021e6cd85fe7bb31b353310ee7d"
server_key_iv: "08119cfec5d88c807e571745d86b55d2"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:34.678650 16388 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.001s
I20260807 15:16:34.704947 16410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:34.707361 16388 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:16:34.707631 16388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "cc35e1fbcc954545a7320603ea1fdf28"
format_stamp: "Formatted at 2026-08-07 15:16:34 on dist-test-slave-gkw7"
server_key: "9c11e021e6cd85fe7bb31b353310ee7d"
server_key_iv: "08119cfec5d88c807e571745d86b55d2"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:34.707880 16388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:34.729987 16388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:34.733937 16388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:34.734167 16388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:34.734957 16388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:16:34.736166 16388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:16:34.736238 16388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:34.736304 16388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:16:34.736338 16388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:34.750640 16569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:42209 every 8 connection(s)
I20260807 15:16:34.750648 16388 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:42209
I20260807 15:16:34.752254 16388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:16:34.758644 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 16388
I20260807 15:16:34.758805 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:16:34.759158 32667 external_mini_cluster.cc:1428] Setting key b63bca0bcce7afd45199311f193ac457
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:16:34.780187 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:34.754516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:60771 (local address 127.31.230.254:41425)
0807 15:16:34.758910 (+ 4394us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:34.758916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:34.765717 (+ 6801us) server_negotiation.cc:407] Connection header received
0807 15:16:34.767149 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:34.767153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:34.767207 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:34.767270 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:34.769077 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.769954 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.771824 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.772197 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.772299 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:34.775635 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:34.775640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:34.775643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:34.775672 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:34.777562 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:34.778378 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:34.778382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:34.778384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:34.778479 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:34.778932 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:34.778935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:34.778937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:34.779162 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:34.779242 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:34.779858 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 15:16:34.779995 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4261}
I20260807 15:16:34.781131 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:34.755110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:41425 (local address 127.31.230.194:60771)
0807 15:16:34.765486 (+ 10376us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:34.765603 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:34.766878 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:34.767749 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:34.767761 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:34.768158 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:34.768895 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:34.768915 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.770134 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.770137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:34.771669 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:34.771682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:34.772532 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:34.772536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:34.772770 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:34.773563 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:34.773589 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:34.775423 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:34.777729 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:34.777735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:34.777754 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:34.778243 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:34.778617 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:34.778620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:34.778622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:34.778812 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:34.779331 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:34.779337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:34.779585 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 15:16:34.779813 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 15:16:34.780155 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10043,"thread_start_us":167,"threads_started":1}
I20260807 15:16:34.782568 16570 heartbeater.cc:347] Connected to a master server at 127.31.230.254:41425
I20260807 15:16:34.782835 16570 heartbeater.cc:464] Registering TS with master...
I20260807 15:16:34.783413 16570 heartbeater.cc:511] Master 127.31.230.254:41425 requested a full tablet report, sending...
I20260807 15:16:34.784617 16078 ts_manager.cc:194] Registered new tserver with Master: cc35e1fbcc954545a7320603ea1fdf28 (127.31.230.194:42209)
I20260807 15:16:34.785410 16078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:60771
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:16:34.812316 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:33107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:41425
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:16:34 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
W20260807 15:16:34.999650 16578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:35.000159 16578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:35.000293 16578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:35.006155 16578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:35.006294 16578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:35.006425 16578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:16:35.013450 16578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.230.212:33107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:41425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:16:35.014825 16578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:35.015965 16578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:35.026151 16587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:35.026566 16590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:35.026562 16588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:35.026814 16578 server_base.cc:1034] running on GCE node
I20260807 15:16:35.027232 16578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:35.027979 16578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:35.029170 16578 hybrid_clock.cc:648] HybridClock initialized: now 1786115795029124 us; error 61 us; skew 500 ppm
Aug 07 15:16:35 dist-test-slave-gkw7 krb5kdc[15446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:35.032346 16578 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:16:35.033390 16578 webserver.cc:460] Webserver started at http://127.31.230.195:39221/ using document root <none> and password file <none>
I20260807 15:16:35.034060 16578 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:35.034159 16578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:35.034412 16578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:16:35 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:16:35.049235 16578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "bd78f36167dc484f8262d6c0184c9b0b"
format_stamp: "Formatted at 2026-08-07 15:16:35 on dist-test-slave-gkw7"
server_key: "a1118e0d3fd21c97a079a00ed7669ec9"
server_key_iv: "234deab1e78efe1d50157ad01d6e90a5"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:35.050136 16578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "bd78f36167dc484f8262d6c0184c9b0b"
format_stamp: "Formatted at 2026-08-07 15:16:35 on dist-test-slave-gkw7"
server_key: "a1118e0d3fd21c97a079a00ed7669ec9"
server_key_iv: "234deab1e78efe1d50157ad01d6e90a5"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:35.055693 16578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 15:16:35.073361 16601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:35.075366 16578 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 15:16:35.075641 16578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "bd78f36167dc484f8262d6c0184c9b0b"
format_stamp: "Formatted at 2026-08-07 15:16:35 on dist-test-slave-gkw7"
server_key: "a1118e0d3fd21c97a079a00ed7669ec9"
server_key_iv: "234deab1e78efe1d50157ad01d6e90a5"
server_key_version: "kuduclusterkey@0"
I20260807 15:16:35.075825 16578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:35.109378 16578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:35.113605 16578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:35.113969 16578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:35.114864 16578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:16:35.116194 16578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:16:35.116304 16578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:35.116408 16578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:16:35.116483 16578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:35.134066 16578 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:40663
I20260807 15:16:35.135551 16578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:16:35.136612 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 16578
I20260807 15:16:35.136749 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:16:35.137120 32667 external_mini_cluster.cc:1428] Setting key 8b3ba42715f836bd8a538a24fd4cb4e3
I20260807 15:16:35.140159 16744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:40663 every 8 connection(s)
Aug 07 15:16:35 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:16:35.161000 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.143326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:34973 (local address 127.31.230.254:41425)
0807 15:16:35.143478 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.143483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.144663 (+ 1180us) server_negotiation.cc:407] Connection header received
0807 15:16:35.146167 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.146170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.146225 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.146279 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:35.148011 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.148859 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.152394 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.152746 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.152831 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.156274 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:35.156279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:35.156282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:35.156312 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:35.158264 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:35.158958 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:35.158962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:35.158964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:35.159063 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:35.159538 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:35.159553 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:35.159555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:35.159809 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:35.159901 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.160674 (+ 773us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.160820 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:16:35.164948 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.143873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:41425 (local address 127.31.230.195:34973)
0807 15:16:35.144488 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.144557 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:35.145885 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:35.146718 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:35.146728 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:35.147128 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:35.147832 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.147853 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.149034 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.149037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.152250 (+ 3213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.152262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.153064 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.153068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.153301 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.154124 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:35.154146 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:35.156045 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:35.158425 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:35.158431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:35.158451 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:35.158829 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:35.159197 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:35.159200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:35.159202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:35.159411 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:35.159978 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:35.159984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:35.160353 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 15:16:35.163352 (+ 2999us) client_negotiation.cc:240] Negotiation successful
0807 15:16:35.163752 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":162,"threads_started":1}
I20260807 15:16:35.166612 16745 heartbeater.cc:347] Connected to a master server at 127.31.230.254:41425
I20260807 15:16:35.166870 16745 heartbeater.cc:464] Registering TS with master...
I20260807 15:16:35.167467 16745 heartbeater.cc:511] Master 127.31.230.254:41425 requested a full tablet report, sending...
I20260807 15:16:35.168673 16078 ts_manager.cc:194] Registered new tserver with Master: bd78f36167dc484f8262d6c0184c9b0b (127.31.230.195:40663)
I20260807 15:16:35.169445 16078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:34973
I20260807 15:16:35.175071 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:16:35.188794 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.177927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.31.230.254:41425)
0807 15:16:35.178051 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.178056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.178785 (+ 729us) server_negotiation.cc:407] Connection header received
0807 15:16:35.178982 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.178986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.179038 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.179111 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:35.180156 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.180925 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.182358 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.182648 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.182736 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.184741 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:35.184745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:35.184748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:35.184776 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:35.186656 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:35.187228 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:35.187232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:35.187234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:35.187326 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:35.187679 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:35.187682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:35.187684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:35.187923 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:35.188034 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.188498 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.188636 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:16:35.193146 16078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:16:35.198035 16078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:16:35.226432 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.216111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:43891 (local address 127.0.0.1:41640)
0807 15:16:35.218492 (+ 2381us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.218503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:35.218568 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:35.220380 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:35.220389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:35.220416 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:16:35.220573 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.220582 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.222290 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.222294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.223767 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.223779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.225842 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.225845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.225964 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.226014 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:16:35.226124 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:16:35.226264 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2327,"thread_start_us":222,"threads_started":1}
I20260807 15:16:35.227562 16773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.216321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.31.230.193:43891)
0807 15:16:35.218888 (+ 2567us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.218897 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.218925 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:16:35.219012 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.219018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.220196 (+ 1178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.220757 (+ 561us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:16:35.220899 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.222144 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.224777 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.225696 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.227105 (+ 1409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.227256 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.227312 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.227418 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2448,"thread_start_us":76,"threads_started":1}
I20260807 15:16:35.236840 16271 tablet_service.cc:1467] Processing CreateTablet for tablet 94fa45c812c849eeaf19f6e29501e395 (DEFAULT_TABLE table=test-table [id=3dd8a37dc6b04bf59a005c04aedd4f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:16:35.238071 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.215628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42209 (local address 127.0.0.1:51952)
0807 15:16:35.218231 (+ 2603us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.218243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:35.218325 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:35.222980 (+ 4655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:35.222983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:35.223002 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:16:35.223100 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.223106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.232440 (+ 9334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.232444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.233717 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.233732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.237608 (+ 3876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.237612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.237724 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.237767 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:16:35.237822 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:16:35.237904 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2539,"thread_start_us":163,"threads_started":1}
I20260807 15:16:35.238402 16271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94fa45c812c849eeaf19f6e29501e395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:35.239144 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.215798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.31.230.194:42209)
0807 15:16:35.221794 (+ 5996us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.221814 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.221841 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:16:35.221924 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.221929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.222156 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.223261 (+ 1105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:16:35.223405 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.224529 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.233961 (+ 9432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.234901 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.238670 (+ 3769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.238840 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.238899 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.239003 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5843,"thread_start_us":128,"threads_started":1}
I20260807 15:16:35.239851 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.212619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:40663 (local address 127.0.0.1:51298)
0807 15:16:35.217751 (+ 5132us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.217812 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:35.217949 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:35.224694 (+ 6745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:35.224697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:35.224718 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:16:35.224818 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.224824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.235064 (+ 10240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.235069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.236300 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.236312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.239473 (+ 3161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.239477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.239575 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.239599 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:16:35.239651 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:16:35.239718 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4898,"thread_start_us":239,"threads_started":1}
I20260807 15:16:35.240774 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.213920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51298 (local address 127.31.230.195:40663)
0807 15:16:35.222494 (+ 8574us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.222505 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.222527 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:16:35.222608 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.222613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.222796 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.224973 (+ 2177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:16:35.225134 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.226260 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.236513 (+ 10253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.237419 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.240219 (+ 2800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.240389 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.240448 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.240619 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8468,"thread_start_us":231,"threads_started":1}
I20260807 15:16:35.244125 16473 tablet_service.cc:1467] Processing CreateTablet for tablet 94fa45c812c849eeaf19f6e29501e395 (DEFAULT_TABLE table=test-table [id=3dd8a37dc6b04bf59a005c04aedd4f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:16:35.244577 16661 tablet_service.cc:1467] Processing CreateTablet for tablet 94fa45c812c849eeaf19f6e29501e395 (DEFAULT_TABLE table=test-table [id=3dd8a37dc6b04bf59a005c04aedd4f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:16:35.245316 16473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94fa45c812c849eeaf19f6e29501e395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:35.245563 16661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94fa45c812c849eeaf19f6e29501e395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:35.251395 16774 tablet_bootstrap.cc:492] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: Bootstrap starting.
I20260807 15:16:35.254575 16774 tablet_bootstrap.cc:654] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:35.255817 16774 log.cc:824] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:35.259958 16774 tablet_bootstrap.cc:492] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: No bootstrap required, opened a new log
I20260807 15:16:35.260200 16774 ts_tablet_manager.cc:1397] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 15:16:35.261641 16777 tablet_bootstrap.cc:492] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Bootstrap starting.
I20260807 15:16:35.261935 16775 tablet_bootstrap.cc:492] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Bootstrap starting.
I20260807 15:16:35.264158 16775 tablet_bootstrap.cc:654] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:35.264673 16777 tablet_bootstrap.cc:654] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:35.265218 16774 raft_consensus.cc:348] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.265570 16777 log.cc:824] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:35.265508 16774 raft_consensus.cc:374] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:16:35.265928 16774 raft_consensus.cc:729] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80ceca5d75245d59411c04149cde4b6, State: Initialized, Role: FOLLOWER
I20260807 15:16:35.266553 16774 consensus_queue.cc:260] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.269086 16774 ts_tablet_manager.cc:1428] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: Time spent starting tablet: real 0.009s user 0.000s sys 0.007s
I20260807 15:16:35.269579 16380 heartbeater.cc:503] Master 127.31.230.254:41425 was elected leader, sending a full tablet report...
I20260807 15:16:35.270184 16775 log.cc:824] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:35.275249 16777 tablet_bootstrap.cc:492] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: No bootstrap required, opened a new log
I20260807 15:16:35.275625 16777 ts_tablet_manager.cc:1397] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20260807 15:16:35.280326 16777 raft_consensus.cc:348] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.280800 16777 raft_consensus.cc:374] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:16:35.280929 16777 raft_consensus.cc:729] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd78f36167dc484f8262d6c0184c9b0b, State: Initialized, Role: FOLLOWER
I20260807 15:16:35.281589 16777 consensus_queue.cc:260] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.284180 16745 heartbeater.cc:503] Master 127.31.230.254:41425 was elected leader, sending a full tablet report...
I20260807 15:16:35.284613 16777 ts_tablet_manager.cc:1428] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20260807 15:16:35.286935 16775 tablet_bootstrap.cc:492] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: No bootstrap required, opened a new log
I20260807 15:16:35.287179 16775 ts_tablet_manager.cc:1397] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.000s
I20260807 15:16:35.294138 16775 raft_consensus.cc:348] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.295183 16775 raft_consensus.cc:374] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:16:35.295333 16775 raft_consensus.cc:729] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc35e1fbcc954545a7320603ea1fdf28, State: Initialized, Role: FOLLOWER
I20260807 15:16:35.296758 16775 consensus_queue.cc:260] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.299003 16775 ts_tablet_manager.cc:1428] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260807 15:16:35.299443 16570 heartbeater.cc:503] Master 127.31.230.254:41425 was elected leader, sending a full tablet report...
Aug 07 15:16:35 dist-test-slave-gkw7 krb5kdc[15446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115794, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
W20260807 15:16:35.391642 16748 tablet.cc:2369] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:16:35.395704 16381 tablet.cc:2369] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:16:35.408069 16781 raft_consensus.cc:482] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:16:35.408347 16781 raft_consensus.cc:504] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.409994 16781 leader_election.cc:288] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd78f36167dc484f8262d6c0184c9b0b (127.31.230.195:40663), cc35e1fbcc954545a7320603ea1fdf28 (127.31.230.194:42209)
I20260807 15:16:35.418768 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.411771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:53995 (local address 127.31.230.195:40663)
0807 15:16:35.411920 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.411925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.413946 (+ 2021us) server_negotiation.cc:407] Connection header received
0807 15:16:35.414250 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.414254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.414324 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.414383 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:16:35.414831 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.415646 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.417349 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.417984 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.418100 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.418159 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.418523 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.418609 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:16:35.419274 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.411781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:37925 (local address 127.31.230.194:42209)
0807 15:16:35.412025 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.412029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.414866 (+ 2837us) server_negotiation.cc:407] Connection header received
0807 15:16:35.415059 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.415062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.415114 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.415168 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:16:35.415650 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.416361 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.417936 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.418564 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.418672 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.418720 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.419068 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.419148 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260807 15:16:35.419498 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.410492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:40663 (local address 127.31.230.193:53995)
0807 15:16:35.413836 (+ 3344us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.413854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:35.414095 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:35.414544 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:35.414547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:35.414565 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:16:35.414689 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.414697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.415839 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.415843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.417197 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.417210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.418273 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.418276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.418367 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.418392 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:16:35.419249 (+ 857us) client_negotiation.cc:240] Negotiation successful
0807 15:16:35.419333 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3233,"thread_start_us":279,"threads_started":1}
I20260807 15:16:35.419919 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.411118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42209 (local address 127.31.230.193:37925)
0807 15:16:35.414756 (+ 3638us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.414773 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:35.414936 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:35.415316 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:35.415319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:35.415336 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:16:35.415452 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.415459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.416515 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.416518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.417787 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:35.417799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.418823 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.418826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:35.418934 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.418953 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:16:35.419694 (+ 741us) client_negotiation.cc:240] Negotiation successful
0807 15:16:35.419775 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3567,"thread_start_us":233,"threads_started":1}
I20260807 15:16:35.420302 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94fa45c812c849eeaf19f6e29501e395" candidate_uuid: "b80ceca5d75245d59411c04149cde4b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd78f36167dc484f8262d6c0184c9b0b" is_pre_election: true
I20260807 15:16:35.420688 16672 raft_consensus.cc:2454] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80ceca5d75245d59411c04149cde4b6 in term 0.
I20260807 15:16:35.420670 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94fa45c812c849eeaf19f6e29501e395" candidate_uuid: "b80ceca5d75245d59411c04149cde4b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc35e1fbcc954545a7320603ea1fdf28" is_pre_election: true
I20260807 15:16:35.420955 16510 raft_consensus.cc:2454] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80ceca5d75245d59411c04149cde4b6 in term 0.
I20260807 15:16:35.421233 16223 leader_election.cc:302] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80ceca5d75245d59411c04149cde4b6, bd78f36167dc484f8262d6c0184c9b0b; no voters:
I20260807 15:16:35.421602 16781 raft_consensus.cc:2781] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:16:35.421725 16781 raft_consensus.cc:482] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:16:35.421797 16781 raft_consensus.cc:3037] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:35.423285 16781 raft_consensus.cc:504] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.423689 16781 leader_election.cc:288] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [CANDIDATE]: Term 1 election: Requested vote from peers bd78f36167dc484f8262d6c0184c9b0b (127.31.230.195:40663), cc35e1fbcc954545a7320603ea1fdf28 (127.31.230.194:42209)
I20260807 15:16:35.424707 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94fa45c812c849eeaf19f6e29501e395" candidate_uuid: "b80ceca5d75245d59411c04149cde4b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc35e1fbcc954545a7320603ea1fdf28"
I20260807 15:16:35.424850 16510 raft_consensus.cc:3037] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:35.426168 16672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94fa45c812c849eeaf19f6e29501e395" candidate_uuid: "b80ceca5d75245d59411c04149cde4b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd78f36167dc484f8262d6c0184c9b0b"
I20260807 15:16:35.426291 16510 raft_consensus.cc:2454] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80ceca5d75245d59411c04149cde4b6 in term 1.
I20260807 15:16:35.426311 16672 raft_consensus.cc:3037] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:35.426699 16224 leader_election.cc:302] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80ceca5d75245d59411c04149cde4b6, cc35e1fbcc954545a7320603ea1fdf28; no voters:
I20260807 15:16:35.426884 16781 raft_consensus.cc:2781] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:16:35.427261 16781 raft_consensus.cc:686] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [term 1 LEADER]: Becoming Leader. State: Replica: b80ceca5d75245d59411c04149cde4b6, State: Running, Role: LEADER
I20260807 15:16:35.427666 16781 consensus_queue.cc:237] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } }
I20260807 15:16:35.427798 16672 raft_consensus.cc:2454] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80ceca5d75245d59411c04149cde4b6 in term 1.
I20260807 15:16:35.431428 16077 catalog_manager.cc:5495] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b80ceca5d75245d59411c04149cde4b6 (127.31.230.193). New cstate: current_term: 1 leader_uuid: "b80ceca5d75245d59411c04149cde4b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80ceca5d75245d59411c04149cde4b6" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 43891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 } health_report { overall_health: UNKNOWN } } }
I20260807 15:16:35.466058 16773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.460889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.31.230.193:43891)
0807 15:16:35.461063 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:35.461068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:35.461085 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:16:35.461180 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:35.461183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:35.461256 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:35.461373 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:35.461882 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.462725 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.464076 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:35.464422 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:35.464539 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:35.464940 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:35.465495 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:35.465699 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:35.465810 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:16:35.465881 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 15:16:35.478057 16510 raft_consensus.cc:1261] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 1 FOLLOWER]: Refusing update from remote peer b80ceca5d75245d59411c04149cde4b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:16:35.478477 16672 raft_consensus.cc:1261] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 1 FOLLOWER]: Refusing update from remote peer b80ceca5d75245d59411c04149cde4b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:16:35.479125 16792 consensus_queue.cc:1035] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc35e1fbcc954545a7320603ea1fdf28" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:16:35.479923 16792 consensus_queue.cc:1035] T 94fa45c812c849eeaf19f6e29501e395 P b80ceca5d75245d59411c04149cde4b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd78f36167dc484f8262d6c0184c9b0b" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 40663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:16:35.502180 16796 mvcc.cc:205] Tried to move back new op lower bound from 7315930298269044736 to 7315930298075230208. Current Snapshot: MvccSnapshot[applied={T|T < 7315930298269044736}]
I20260807 15:16:35.506491 16798 mvcc.cc:205] Tried to move back new op lower bound from 7315930298269044736 to 7315930298075230208. Current Snapshot: MvccSnapshot[applied={T|T < 7315930298269044736}]
I20260807 15:16:35.514886 16077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:16:35.515101 16077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:16:35.518522 16077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b107eadb1bf847dc80f3a28ce8918fd8: Sending DeleteTablet for 3 replicas of tablet 94fa45c812c849eeaf19f6e29501e395
I20260807 15:16:35.519805 16473 tablet_service.cc:1514] Processing DeleteTablet for tablet 94fa45c812c849eeaf19f6e29501e395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:16:35 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:51952
I20260807 15:16:35.520318 16661 tablet_service.cc:1514] Processing DeleteTablet for tablet 94fa45c812c849eeaf19f6e29501e395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:16:35 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:51298
I20260807 15:16:35.520690 16806 tablet_replica.cc:330] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: stopping tablet replica
I20260807 15:16:35.521150 16806 raft_consensus.cc:2229] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:16:35.521093 16271 tablet_service.cc:1514] Processing DeleteTablet for tablet 94fa45c812c849eeaf19f6e29501e395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:16:35 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:41640
I20260807 15:16:35.521663 16806 raft_consensus.cc:2258] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:16:35.522292 16807 tablet_replica.cc:330] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: stopping tablet replica
I20260807 15:16:35.522646 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 16200
I20260807 15:16:35.534127 16806 ts_tablet_manager.cc:1905] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:16:35.536975 16806 ts_tablet_manager.cc:1918] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:16:35.537130 16806 log.cc:1196] T 94fa45c812c849eeaf19f6e29501e395 P cc35e1fbcc954545a7320603ea1fdf28: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/94fa45c812c849eeaf19f6e29501e395
I20260807 15:16:35.538499 16807 raft_consensus.cc:2229] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:16:35.538913 16807 raft_consensus.cc:2258] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 15:16:35.546276 16059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:43891 (error 108)
I20260807 15:16:35.546880 16807 ts_tablet_manager.cc:1905] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:16:35.547641 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 16388
I20260807 15:16:35.566787 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.566333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:43891 (local address 127.0.0.1:41652)
0807 15:16:35.566577 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.566672 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260807 15:16:35.567107 16059 catalog_manager.cc:4584] TS b80ceca5d75245d59411c04149cde4b6 (127.31.230.193:43891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94fa45c812c849eeaf19f6e29501e395: Network error: Client connection negotiation failed: client connection to 127.31.230.193:43891: connect: Connection refused (error 111)
I20260807 15:16:35.594621 16807 ts_tablet_manager.cc:1918] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:16:35.594786 16807 log.cc:1196] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/94fa45c812c849eeaf19f6e29501e395
I20260807 15:16:35.595126 16807 ts_tablet_manager.cc:1939] T 94fa45c812c849eeaf19f6e29501e395 P bd78f36167dc484f8262d6c0184c9b0b: Deleting consensus metadata
I20260807 15:16:35.596589 16059 catalog_manager.cc:4841] TS bd78f36167dc484f8262d6c0184c9b0b (127.31.230.195:40663): tablet 94fa45c812c849eeaf19f6e29501e395 (table test-table [id=3dd8a37dc6b04bf59a005c04aedd4f16]) successfully deleted
I20260807 15:16:35.599362 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 16578
I20260807 15:16:35.601835 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.599565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:42209 (local address 127.0.0.1:51962)
0807 15:16:35.601602 (+ 2037us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.601689 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.194:42209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1895}
I20260807 15:16:35.622745 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.622433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:43891 (local address 127.0.0.1:41662)
0807 15:16:35.622586 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.622657 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 15:16:35.623660 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:35.623424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:43891 (local address 127.0.0.1:41678)
0807 15:16:35.623537 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:35.623593 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 15:16:35.628248 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 16042
I20260807 15:16:35.641764 32667 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:16:35.987712 32667 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:16:35.987879 32667 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:16:36.344134 32667 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:16:36.344 UTC [15487] LOG: received smart shutdown request
2026-08-07 15:16:36.348 UTC [15487] LOG: background worker "logical replication launcher" (PID 15499) exited with exit code 1
2026-08-07 15:16:36.349 UTC [15492] LOG: shutting down
2026-08-07 15:16:36.520 UTC [15487] LOG: database system is shut down
2026-08-07T15:16:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50181 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:16:36 dist-test-slave-gkw7 krb5kdc[16832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:16:36 dist-test-slave-gkw7 krb5kdc[16832](info): set up 2 sockets
Aug 07 15:16:36 dist-test-slave-gkw7 krb5kdc[16832](info): commencing operation
krb5kdc: starting...
W20260807 15:16:37.902338 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:16:37 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:16:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/rangeradmin_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/rangeradmin_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/rangerlookup_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/rangerlookup_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
I20260807 15:16:38.092352 32667 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLoCTfz/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/postgres -l logfile start
2026-08-07 15:16:40.730 UTC [16870] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:16:40.730 UTC [16870] LOG: listening on IPv4 address "127.31.230.212", port 60353
2026-08-07 15:16:40.734 UTC [16870] LOG: listening on Unix socket "/tmp/.s.PGSQL.60353"
2026-08-07 15:16:40.757 UTC [16874] LOG: database system was shut down at 2026-08-07 15:16:39 UTC
2026-08-07 15:16:40.761 UTC [16870] LOG: database system is ready to accept connections
I20260807 15:16:41.930215 32667 mini_postgres.cc:96] Postgres bound to 60353
2026-08-07 15:16:41.938 UTC [16885] FATAL: database "slave" does not exist
127.31.230.212:60353 - accepting connections
I20260807 15:16:41.938804 32667 mini_ranger.cc:161] Starting Ranger...
I20260807 15:16:41.956326 32667 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:16:42.478956 32667 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:16:42.479061 32667 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-admin
2026-08-07 15:16:42,776 [I] DB FLAVOR :POSTGRES
2026-08-07 15:16:42,777 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:16:42,777 [I] Checking connection..
2026-08-07 15:16:42,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:16:43,031 [I] Checking connection passed.
2026-08-07 15:16:43,032 [I] --------- Verifying version history table ---------
2026-08-07 15:16:43,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:16:43,307 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:16:43,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:16:43,583 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:16:43,584 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:16:43,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:16:43.845 UTC [16978] WARNING: there is no transaction in progress
2026-08-07 15:16:43,853 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:16:43,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:16:44,127 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:16:44,128 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:16:44,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:16:44,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:16:44,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-07 15:16:44,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:16:45,204 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:16:45,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:16:45,477 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:16:45,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:16:45,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:16:45,995 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:16:45,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:16:46.994 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.011 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.029 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.046 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.063 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.082 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.138 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.145 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.161 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.172 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.182 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.196 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.206 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.212 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.219 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47.226 UTC [17129] WARNING: there is no transaction in progress
2026-08-07 15:16:47,898 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:16:47,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:16:48,155 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:16:48,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:16:48,411 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:16:48,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:16:48,680 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:16:48,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:16:48,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:16:49,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-07 15:16:49,464 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:16:49,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:16:49,727 [I] DB_PATCHES have already been applied
I20260807 15:16:49.733144 32667 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLoCTfz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:16:49.733211 32667 mini_ranger.cc:193] Using host: 127.31.230.212
I20260807 15:16:49.738142 32667 mini_ranger.cc:227] Ranger admin URL: http://127.31.230.212:60377
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:16:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:16:51.112555 32667 mini_ranger.cc:160] Time spent starting Ranger: real 9.174s user 0.000s sys 0.013s
Aug 07, 2026 3:17:10 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:17:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 3:17:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:17:10 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:17:11 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:17:12.069084 32667 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.230.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.230.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/rangerkms_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.230.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/rangerkms_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.230.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.230.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.230.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/HTTP_127.31.230.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:17:12 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115832, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:17:12.162789 32667 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:17:12.174050 32667 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:17:13.062065 32667 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:17:13.062184 32667 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms
I20260807 15:17:13.062209 32667 mini_ranger_kms.cc:227] Using postgres at 127.31.230.212:60353
2026-08-07 15:17:13,260 [I] DB FLAVOR :POSTGRES
2026-08-07 15:17:13,260 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:17:13,260 [I] Checking connection
2026-08-07 15:17:13,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:17:13,527 [I] connection success
2026-08-07 15:17:13,527 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:17:13,528 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:17:13,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:17:13,771 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:17:13,771 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:17:13,771 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:17:13,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.230.212:60353/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:17:14,034 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:17:14.487844 32667 mini_ranger_kms.cc:310] Created kms service
I20260807 15:17:14.667128 32667 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:17:14.720999 32667 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:17:14 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115832, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:15.130847 32667 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:17:15.131014 32667 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLoCTfz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLoCTfz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:17:15.131038 32667 mini_ranger_kms.cc:240] Using host: 127.31.230.212
I20260807 15:17:15.136073 32667 mini_ranger_kms.cc:276] Ranger KMS PID: 17386
I20260807 15:17:15.136163 32667 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.230.212:53789
Aug 07, 2026 3:17:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:17:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:17:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:17:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:17:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:17:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:17:17 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.230.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:17.818257 32667 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.655s user 0.026s sys 0.158s
I20260807 15:17:17.818408 32667 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:17:17.818506 32667 mini_ranger_kms.cc:401] 127.31.230.212:53789/kms/v1/keys
Aug 07 15:17:17 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.230.212@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
Aug 07 15:17:18 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.230.212@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
Aug 07 15:17:20 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:17:20.493688 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34251
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:34251
--ranger_config_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:20.627179 17457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:20.627460 17457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:20.627501 17457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:20.631531 17457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:20.631605 17457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:20.631624 17457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:20.631641 17457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:20.631682 17457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:20.637830 17457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.230.212:53789/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:34251
--ranger_config_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:20.639039 17457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:20.640141 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:20.649551 17468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:20.649662 17465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:20.649591 17463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:20.649636 17457 server_base.cc:1034] running on GCE node
I20260807 15:17:20.650600 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:20.652140 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:20.653352 17457 hybrid_clock.cc:648] HybridClock initialized: now 1786115840653335 us; error 35 us; skew 500 ppm
Aug 07 15:17:20 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115840, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:20.659647 17457 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:20.660683 17457 webserver.cc:460] Webserver started at http://127.31.230.254:33157/ using document root <none> and password file <none>
I20260807 15:17:20.661275 17457 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:20.661360 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:20.661659 17457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:17:20 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115840, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:20.719740 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "87e4457530e44045b87bc4cd20c389be"
format_stamp: "Formatted at 2026-08-07 15:17:20 on dist-test-slave-gkw7"
server_key: "2e405d7f88e05d898794da58e00f1635"
server_key_iv: "275f4c858f67b60703201f07a3313613"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:20.720435 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "87e4457530e44045b87bc4cd20c389be"
format_stamp: "Formatted at 2026-08-07 15:17:20 on dist-test-slave-gkw7"
server_key: "2e405d7f88e05d898794da58e00f1635"
server_key_iv: "275f4c858f67b60703201f07a3313613"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:20.724964 17457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 15:17:20.744345 17478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:20.745961 17457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:20.746143 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "87e4457530e44045b87bc4cd20c389be"
format_stamp: "Formatted at 2026-08-07 15:17:20 on dist-test-slave-gkw7"
server_key: "2e405d7f88e05d898794da58e00f1635"
server_key_iv: "275f4c858f67b60703201f07a3313613"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:20.746309 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:20.761483 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:20.764353 17457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:20.764580 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:20.772102 17457 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:34251
I20260807 15:17:20.772131 17549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:34251 every 8 connection(s)
I20260807 15:17:20.773082 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:20.774108 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 17457
I20260807 15:17:20.774272 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:17:20.774554 32667 external_mini_cluster.cc:1428] Setting key 046a7755a2ca77a3adbef072ca253c1f
I20260807 15:17:20.776333 17551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:20.783075 17551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Bootstrap starting.
Aug 07 15:17:20 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:20.785359 17551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:20.786521 17551 log.cc:824] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:20.788566 17551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: No bootstrap required, opened a new log
I20260807 15:17:20.789031 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:20.775940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.31.230.254:34251)
0807 15:17:20.776353 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:20.776372 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:20.776418 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:17:20.777085 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:20.777102 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:20.777428 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:20.777787 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:20.778730 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:20.779714 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:20.780888 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:20.781341 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:20.781457 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:20.784377 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:20.784382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:20.784395 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:20.784422 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:20.786160 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:20.786621 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:20.786628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:20.786633 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:20.786782 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:20.787076 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:20.787080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:20.787082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:20.787478 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:20.787686 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:20.787990 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 15:17:20.788266 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":142,"threads_started":1}
I20260807 15:17:20.792073 17551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:20.792321 17551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:20.792414 17551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e4457530e44045b87bc4cd20c389be, State: Initialized, Role: FOLLOWER
I20260807 15:17:20.792915 17551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:20.793083 17551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:20.793156 17551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:20.793272 17551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:20.794219 17551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:20.794567 17551 leader_election.cc:302] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87e4457530e44045b87bc4cd20c389be; no voters:
I20260807 15:17:20.794878 17551 leader_election.cc:288] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:17:20.795003 17557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:20.795284 17557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 LEADER]: Becoming Leader. State: Replica: 87e4457530e44045b87bc4cd20c389be, State: Running, Role: LEADER
I20260807 15:17:20.795643 17557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:20.796275 17551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:20.797402 17560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e4457530e44045b87bc4cd20c389be. Latest consensus state: current_term: 1 leader_uuid: "87e4457530e44045b87bc4cd20c389be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } } }
I20260807 15:17:20.797564 17560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:20.797899 17558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87e4457530e44045b87bc4cd20c389be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } } }
I20260807 15:17:20.797968 17567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:20.798004 17558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:20.800763 17567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:20.801295 17551 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:17:20.811098 17567 catalog_manager.cc:1347] Generated new cluster ID: 2cd9eb4f9816411eb77d6d73c3c2c638
I20260807 15:17:20.811187 17567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:20.839296 17567 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:17:20.840771 17567 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:20.851181 17567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Generated new TSK 0
I20260807 15:17:20.852067 17567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:17:21.572456 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:34251
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:21.726727 17615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:21.727105 17615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:21.727173 17615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:21.733181 17615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:21.733273 17615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:21.733362 17615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:17:21.740803 17615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.230.212:53789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:34251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:21.742070 17615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:21.743185 17615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:21.754750 17625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:21.754976 17615 server_base.cc:1034] running on GCE node
W20260807 15:17:21.755359 17624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:21.755605 17627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:21.756101 17615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:21.756870 17615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:21.764283 17615 hybrid_clock.cc:648] HybridClock initialized: now 1786115841764195 us; error 102 us; skew 500 ppm
Aug 07 15:17:21 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115841, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:21.768258 17615 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:17:21.769594 17615 webserver.cc:460] Webserver started at http://127.31.230.193:38827/ using document root <none> and password file <none>
I20260807 15:17:21.770422 17615 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:21.770581 17615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:21.770916 17615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:17:21 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115841, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:21.789209 17615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "86637fbf953e46ab9f95c87639fe837e"
format_stamp: "Formatted at 2026-08-07 15:17:21 on dist-test-slave-gkw7"
server_key: "e2dd37f706638e3bde2a9a7172448849"
server_key_iv: "bc42903695d3e1119366ba7fff3c14b2"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:21.790530 17615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "86637fbf953e46ab9f95c87639fe837e"
format_stamp: "Formatted at 2026-08-07 15:17:21 on dist-test-slave-gkw7"
server_key: "e2dd37f706638e3bde2a9a7172448849"
server_key_iv: "bc42903695d3e1119366ba7fff3c14b2"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:21.796767 17615 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 15:17:21.829835 17637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:21.831403 17615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:17:21.831561 17615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "86637fbf953e46ab9f95c87639fe837e"
format_stamp: "Formatted at 2026-08-07 15:17:21 on dist-test-slave-gkw7"
server_key: "e2dd37f706638e3bde2a9a7172448849"
server_key_iv: "bc42903695d3e1119366ba7fff3c14b2"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:21.831730 17615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:21.855784 17615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:21.859365 17615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:21.859584 17615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:21.860291 17615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:21.861331 17615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:21.861423 17615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:21.861498 17615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:21.861577 17615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:21.873234 17615 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:32899
I20260807 15:17:21.873268 17797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:32899 every 8 connection(s)
I20260807 15:17:21.874414 17615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:17:21.874820 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 17615
I20260807 15:17:21.874954 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:17:21.875270 32667 external_mini_cluster.cc:1428] Setting key c8f71ddd2c49a411f400b05b586ea263
Aug 07 15:17:21 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115841, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Aug 07 15:17:21 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115841, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:21.899883 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:21.878634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46593 (local address 127.31.230.254:34251)
0807 15:17:21.878937 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:21.878943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:21.884729 (+ 5786us) server_negotiation.cc:407] Connection header received
0807 15:17:21.886255 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:21.886260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:21.886322 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:21.886391 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:21.888305 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:21.889193 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:21.891037 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:21.891420 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:21.891510 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:21.895147 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:21.895153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:21.895157 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:21.895189 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:21.897157 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:21.897895 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:21.897899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:21.897901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:21.898000 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:21.898438 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:21.898442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:21.898444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:21.898698 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:21.898783 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:21.899521 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 15:17:21.899683 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260807 15:17:21.901644 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:21.877499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34251 (local address 127.31.230.193:46593)
0807 15:17:21.884504 (+ 7005us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:21.884593 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:21.885955 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:21.886928 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:21.886939 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:21.887327 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:21.888099 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:21.888124 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:21.889376 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:21.889380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:21.890879 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:21.890891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:21.892073 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:21.892077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:21.892299 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:21.893041 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:21.893065 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:21.894906 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:21.897318 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:21.897324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:21.897344 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:21.897768 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:21.898121 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:21.898124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:21.898126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:21.898320 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:21.898855 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:21.898862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:21.899217 (+ 355us) client_negotiation.cc:769] Sending connection context
0807 15:17:21.900089 (+ 872us) client_negotiation.cc:240] Negotiation successful
0807 15:17:21.900472 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6685,"thread_start_us":152,"threads_started":1}
I20260807 15:17:21.903468 17800 heartbeater.cc:347] Connected to a master server at 127.31.230.254:34251
I20260807 15:17:21.903736 17800 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:21.904340 17800 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:21.906282 17502 ts_manager.cc:194] Registered new tserver with Master: 86637fbf953e46ab9f95c87639fe837e (127.31.230.193:32899)
I20260807 15:17:21.907889 17502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:46593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:17:21.924536 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:34251
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:22.085398 17809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:22.086305 17809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:22.086447 17809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:22.092696 17809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:22.092782 17809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:22.092877 17809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:17:22.101094 17809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.230.212:53789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:34251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:22.102839 17809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:22.104254 17809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:22.117187 17817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:22.117193 17816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:22.117748 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:22.117954 17809 server_base.cc:1034] running on GCE node
I20260807 15:17:22.118443 17809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:22.119259 17809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:22.120471 17809 hybrid_clock.cc:648] HybridClock initialized: now 1786115842120447 us; error 43 us; skew 500 ppm
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:22.124253 17809 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:17:22.125327 17809 webserver.cc:460] Webserver started at http://127.31.230.194:38611/ using document root <none> and password file <none>
I20260807 15:17:22.125993 17809 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:22.126086 17809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:22.126320 17809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:22.142047 17809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "b4b26409153b494cba87130cdbb14424"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "baae5cf10299b23cd59da3b064c089b7"
server_key_iv: "6dc200be5a327e9e3edb74e4beddd97a"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.142910 17809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "b4b26409153b494cba87130cdbb14424"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "baae5cf10299b23cd59da3b064c089b7"
server_key_iv: "6dc200be5a327e9e3edb74e4beddd97a"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.150234 17809 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260807 15:17:22.169587 17827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.171368 17809 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:17:22.171523 17809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "b4b26409153b494cba87130cdbb14424"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "baae5cf10299b23cd59da3b064c089b7"
server_key_iv: "6dc200be5a327e9e3edb74e4beddd97a"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.171674 17809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:22.186269 17809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:22.190061 17809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:22.190326 17809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:22.191109 17809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:22.192224 17809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:22.192351 17809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.192457 17809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:22.192526 17809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.204140 17809 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:45973
I20260807 15:17:22.207553 17809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:17:22.209565 17983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:45973 every 8 connection(s)
I20260807 15:17:22.215546 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 17809
I20260807 15:17:22.215678 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:17:22.216018 32667 external_mini_cluster.cc:1428] Setting key 908476db28b39816ffb7899a4eeaa39d
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:22.242158 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:22.213063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:45493 (local address 127.31.230.254:34251)
0807 15:17:22.213225 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:22.213231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:22.215047 (+ 1816us) server_negotiation.cc:407] Connection header received
0807 15:17:22.222284 (+ 7237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:22.222288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:22.222347 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:22.222405 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:22.224203 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.225030 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.226829 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.227169 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.227259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:22.233057 (+ 5798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:22.233062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:22.233065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:22.233099 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:22.235080 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:22.235912 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:22.235916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:22.235918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:22.236018 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:22.236483 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:22.236486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:22.236488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:22.236777 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:22.236874 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:22.241747 (+ 4873us) server_negotiation.cc:299] Negotiation successful
0807 15:17:22.241946 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:17:22.244393 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:22.211908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34251 (local address 127.31.230.194:45493)
0807 15:17:22.214858 (+ 2950us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:22.214920 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:22.221976 (+ 7056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:22.222880 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:22.222890 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:22.223288 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:22.224019 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:22.224041 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.225207 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.225210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:22.226668 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:22.226680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.227486 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.227490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:22.227712 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:22.228475 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:22.228497 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:22.232792 (+ 4295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:22.235244 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:22.235251 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:22.235272 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:22.235779 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:22.236144 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:22.236147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:22.236149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:22.236364 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:22.236980 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:22.236987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:22.237357 (+ 370us) client_negotiation.cc:769] Sending connection context
0807 15:17:22.242707 (+ 5350us) client_negotiation.cc:240] Negotiation successful
0807 15:17:22.243150 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2707,"thread_start_us":157,"threads_started":1}
I20260807 15:17:22.246289 17986 heartbeater.cc:347] Connected to a master server at 127.31.230.254:34251
I20260807 15:17:22.246578 17986 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:22.247233 17986 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:22.248539 17502 ts_manager.cc:194] Registered new tserver with Master: b4b26409153b494cba87130cdbb14424 (127.31.230.194:45973)
I20260807 15:17:22.249363 17502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:45493
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:17:22.287205 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:34251
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115841, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
W20260807 15:17:22.462244 17994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:22.462549 17994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:22.462680 17994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:22.467501 17994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:22.467598 17994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:22.467701 17994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:17:22.473287 17994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.230.212:53789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:34251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:22.474512 17994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:22.475641 17994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:22.486414 18005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:22.486392 18001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:22.486392 18000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:22.487211 17994 server_base.cc:1034] running on GCE node
I20260807 15:17:22.487686 17994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:22.488463 17994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:22.489697 17994 hybrid_clock.cc:648] HybridClock initialized: now 1786115842489629 us; error 83 us; skew 500 ppm
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:22.493570 17994 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:17:22.494681 17994 webserver.cc:460] Webserver started at http://127.31.230.195:40913/ using document root <none> and password file <none>
I20260807 15:17:22.495395 17994 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:22.495481 17994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:22.495708 17994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:22.514680 17994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "6cbcf961216c41a68ad9749d5dd265be"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "7f473151365ccc244e74a2237dbf65b3"
server_key_iv: "49c7da6b9413120e803057bb6a024e92"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.515440 17994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "6cbcf961216c41a68ad9749d5dd265be"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "7f473151365ccc244e74a2237dbf65b3"
server_key_iv: "49c7da6b9413120e803057bb6a024e92"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.521133 17994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 15:17:22.542127 18017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.543797 17994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:17:22.543936 17994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "6cbcf961216c41a68ad9749d5dd265be"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "7f473151365ccc244e74a2237dbf65b3"
server_key_iv: "49c7da6b9413120e803057bb6a024e92"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.544091 17994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:22.563730 17994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:22.567503 17994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:22.567723 17994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:22.568437 17994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:22.569471 17994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:22.569543 17994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.569617 17994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:22.569643 17994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.580183 17994 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:37845
I20260807 15:17:22.581362 17994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:17:22.583300 18174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:37845 every 8 connection(s)
I20260807 15:17:22.590726 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 17994
I20260807 15:17:22.590873 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:17:22.591145 32667 external_mini_cluster.cc:1428] Setting key 556d1b7b1c76e60e645e880957954f99
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:22.600383 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:22.585256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:54143 (local address 127.31.230.254:34251)
0807 15:17:22.585409 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:22.585414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:22.586649 (+ 1235us) server_negotiation.cc:407] Connection header received
0807 15:17:22.587927 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:22.587932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:22.587993 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:22.588114 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:22.589568 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.590454 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.592241 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.592576 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.592673 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:22.595907 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:22.595911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:22.595914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:22.595942 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:22.597764 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:22.598442 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:22.598446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:22.598448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:22.598547 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:22.599015 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:22.599018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:22.599020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:22.599259 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:22.599362 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:22.600079 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 15:17:22.600213 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:17:22.601661 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:22.585858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34251 (local address 127.31.230.195:54143)
0807 15:17:22.586459 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:22.586520 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:22.587649 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:22.588306 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:22.588315 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:22.588692 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:22.589355 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:22.589374 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.590621 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.590624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:22.592066 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:22.592092 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:22.592890 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:22.592894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:22.593118 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:22.593888 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:22.593910 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:22.595699 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:22.597896 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:22.597902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:22.597921 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:22.598319 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:22.598675 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:22.598678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:22.598680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:22.598890 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:22.599396 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:22.599404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:22.599766 (+ 362us) client_negotiation.cc:769] Sending connection context
0807 15:17:22.600013 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 15:17:22.600409 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":166,"threads_started":1}
I20260807 15:17:22.603037 18175 heartbeater.cc:347] Connected to a master server at 127.31.230.254:34251
I20260807 15:17:22.603331 18175 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:22.603960 18175 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:22.605221 17502 ts_manager.cc:194] Registered new tserver with Master: 6cbcf961216c41a68ad9749d5dd265be (127.31.230.195:37845)
I20260807 15:17:22.606035 17502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:54143
I20260807 15:17:22.606571 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:17:22.607384 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 17615
I20260807 15:17:22.632171 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 17809
I20260807 15:17:22.656311 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 17994
I20260807 15:17:22.678149 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 17457
I20260807 15:17:22.693253 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34251
--webserver_interface=127.31.230.254
--webserver_port=33157
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:34251
--ranger_config_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:22.806051 18181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:22.806352 18181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:22.806393 18181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:22.810343 18181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:22.810395 18181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:22.810415 18181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:22.810431 18181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:22.810446 18181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:22.815801 18181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:34251
--ranger_config_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=33157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:22.816826 18181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:22.817840 18181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:17:22.826784 18181 server_base.cc:1034] running on GCE node
W20260807 15:17:22.826716 18191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:22.826716 18193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:22.826773 18189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:22.827500 18181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:22.828653 18181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:22.829814 18181 hybrid_clock.cc:648] HybridClock initialized: now 1786115842829798 us; error 33 us; skew 500 ppm
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:22.832656 18181 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:22.833494 18181 webserver.cc:460] Webserver started at http://127.31.230.254:33157/ using document root <none> and password file <none>
I20260807 15:17:22.834054 18181 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:22.834125 18181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:17:22 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:22.897234 18181 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:22.899557 18203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:22.901021 18181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:17:22.901134 18181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "87e4457530e44045b87bc4cd20c389be"
format_stamp: "Formatted at 2026-08-07 15:17:20 on dist-test-slave-gkw7"
server_key: "2e405d7f88e05d898794da58e00f1635"
server_key_iv: "275f4c858f67b60703201f07a3313613"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:22.901764 18181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:22.935878 18181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:22.939086 18181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:22.939260 18181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:22.946970 18181 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:34251
I20260807 15:17:22.947001 18278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:34251 every 8 connection(s)
I20260807 15:17:22.948078 18181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:22.951615 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 18181
I20260807 15:17:22.952075 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:32899
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:34251
--webserver_port=38827
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:22.952690 18280 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:17:22.954483 18280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Bootstrap starting.
I20260807 15:17:22.971284 18280 log.cc:824] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:22.978858 18280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:22.979358 18280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Bootstrap complete.
I20260807 15:17:22.983785 18280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:22.984233 18280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e4457530e44045b87bc4cd20c389be, State: Initialized, Role: FOLLOWER
I20260807 15:17:22.984735 18280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:22.984884 18280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:22.984973 18280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:22.985102 18280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:17:22.987180 18280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:22.987530 18280 leader_election.cc:302] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87e4457530e44045b87bc4cd20c389be; no voters:
I20260807 15:17:22.987911 18280 leader_election.cc:288] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:17:22.988040 18284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:17:22.988266 18284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [term 2 LEADER]: Becoming Leader. State: Replica: 87e4457530e44045b87bc4cd20c389be, State: Running, Role: LEADER
I20260807 15:17:22.988576 18284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } }
I20260807 15:17:22.989207 18280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:22.989552 18284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e4457530e44045b87bc4cd20c389be. Latest consensus state: current_term: 2 leader_uuid: "87e4457530e44045b87bc4cd20c389be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } } }
I20260807 15:17:22.989686 18284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:22.990103 18285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "87e4457530e44045b87bc4cd20c389be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e4457530e44045b87bc4cd20c389be" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34251 } } }
I20260807 15:17:22.990149 18288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:22.991495 18285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:22.994284 18288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:22.994835 18280 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:17:22.994933 18288 catalog_manager.cc:1259] Loaded cluster ID: 2cd9eb4f9816411eb77d6d73c3c2c638
I20260807 15:17:22.995002 18288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:22.997364 18288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:22.998040 18288 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Loaded TSK: 0
I20260807 15:17:22.999261 18288 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:17:23.086755 18282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:23.087045 18282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:23.087123 18282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:23.092072 18282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:23.092154 18282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:23.092255 18282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:17:23.098048 18282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:32899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=38827
--tserver_master_addrs=127.31.230.254:34251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:23.099280 18282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:23.100380 18282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:23.113441 18331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:23.113695 18335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:23.114143 18332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:23.114686 18282 server_base.cc:1034] running on GCE node
I20260807 15:17:23.115185 18282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:23.115885 18282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:23.117103 18282 hybrid_clock.cc:648] HybridClock initialized: now 1786115843117036 us; error 78 us; skew 500 ppm
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:23.120040 18282 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:17:23.121088 18282 webserver.cc:460] Webserver started at http://127.31.230.193:38827/ using document root <none> and password file <none>
I20260807 15:17:23.121732 18282 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:23.121856 18282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:23.153362 18282 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:17:23.161700 18349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.163564 18282 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260807 15:17:23.163787 18282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "86637fbf953e46ab9f95c87639fe837e"
format_stamp: "Formatted at 2026-08-07 15:17:21 on dist-test-slave-gkw7"
server_key: "e2dd37f706638e3bde2a9a7172448849"
server_key_iv: "bc42903695d3e1119366ba7fff3c14b2"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:23.164654 18282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:23.184497 18282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:23.187784 18282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:23.188014 18282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:23.188863 18282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:23.189920 18282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:23.189980 18282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.190042 18282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:23.190109 18282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.204186 18282 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:32899
I20260807 15:17:23.204587 18511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:32899 every 8 connection(s)
I20260807 15:17:23.205673 18282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:17:23.213204 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 18282
I20260807 15:17:23.213768 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:45973
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:34251
--webserver_port=38611
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:23.226964 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.208673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:51665 (local address 127.31.230.254:34251)
0807 15:17:23.209267 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:23.209291 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:23.209763 (+ 472us) server_negotiation.cc:407] Connection header received
0807 15:17:23.211098 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:23.211109 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:23.211416 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:23.211758 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:23.212826 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.214418 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.216108 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.216589 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.216774 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.219784 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:23.219793 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:23.219814 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:23.219855 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:23.222290 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.223032 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.223039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.223048 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.223208 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.223707 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.223713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.223717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.224278 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:23.224405 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:23.225299 (+ 894us) server_negotiation.cc:299] Negotiation successful
0807 15:17:23.225683 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":140,"threads_started":1}
I20260807 15:17:23.228847 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.208784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34251 (local address 127.31.230.193:51665)
0807 15:17:23.209582 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:23.209635 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:23.210571 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:23.211822 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:23.211830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:23.212178 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:23.212638 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:23.212652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.214588 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.214595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:23.215970 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:23.215980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.216772 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.216776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:23.216967 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.217648 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:23.217675 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:23.219578 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:23.222447 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:23.222456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:23.222478 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:23.222868 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:23.223368 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:23.223373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:23.223378 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:23.223576 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:23.224507 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:23.224516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:23.224869 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 15:17:23.227379 (+ 2510us) client_negotiation.cc:240] Negotiation successful
0807 15:17:23.227755 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":154,"threads_started":1}
I20260807 15:17:23.232236 18512 heartbeater.cc:347] Connected to a master server at 127.31.230.254:34251
I20260807 15:17:23.232517 18512 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.233174 18512 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
W20260807 15:17:23.235507 18512 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:34251 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:17:23.235636 18512 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.235826 18512 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:23.236634 18512 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.236832 18512 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
W20260807 15:17:23.237617 18512 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:17:23.377894 18518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:23.378341 18518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:23.378414 18518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:23.384795 18518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:23.384893 18518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:23.384989 18518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:17:23.393455 18518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:45973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=38611
--tserver_master_addrs=127.31.230.254:34251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:23.395021 18518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:23.396502 18518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:23.411176 18526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:23.411317 18528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:23.411227 18525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:23.412586 18518 server_base.cc:1034] running on GCE node
I20260807 15:17:23.413053 18518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:23.413800 18518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:23.414976 18518 hybrid_clock.cc:648] HybridClock initialized: now 1786115843414940 us; error 52 us; skew 500 ppm
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:23.418478 18518 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:17:23.419572 18518 webserver.cc:460] Webserver started at http://127.31.230.194:38611/ using document root <none> and password file <none>
I20260807 15:17:23.420310 18518 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:23.420428 18518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:23.443289 18518 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 15:17:23.446190 18541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.448112 18518 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:17:23.448364 18518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "b4b26409153b494cba87130cdbb14424"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "baae5cf10299b23cd59da3b064c089b7"
server_key_iv: "6dc200be5a327e9e3edb74e4beddd97a"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:23.449244 18518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:23.490571 18518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:23.495375 18518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:23.495699 18518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:23.496615 18518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:23.497936 18518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:23.498018 18518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.498092 18518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:23.498139 18518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.517227 18518 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:45973
I20260807 15:17:23.517668 18700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:45973 every 8 connection(s)
I20260807 15:17:23.518858 18518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:17:23.527281 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 18518
I20260807 15:17:23.527918 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.230.212:53789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:37845
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:34251
--webserver_port=40913
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:23.538470 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.521441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59057 (local address 127.31.230.254:34251)
0807 15:17:23.521700 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:23.521706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:23.522890 (+ 1184us) server_negotiation.cc:407] Connection header received
0807 15:17:23.524406 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:23.524410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:23.524470 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:23.524575 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:23.526343 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.527198 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.528957 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.529338 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.529421 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.533251 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:23.533256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:23.533260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:23.533301 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:23.535355 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.536112 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.536117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.536119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.536241 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.536761 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.536766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.536768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.537083 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:23.537229 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:23.538031 (+ 802us) server_negotiation.cc:299] Negotiation successful
0807 15:17:23.538202 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260807 15:17:23.540809 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.522009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34251 (local address 127.31.230.194:59057)
0807 15:17:23.522681 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:23.522746 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:23.524103 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:23.524845 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:23.524857 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:23.525318 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:23.526129 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:23.526157 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.527376 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.527380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:23.528796 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:23.528807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.529667 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.529671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:23.529873 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.530587 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:23.530609 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:23.533007 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:23.535503 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:23.535514 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:23.535536 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:23.535942 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:23.536373 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:23.536379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:23.536384 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:23.536604 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:23.537230 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:23.537247 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:23.537660 (+ 413us) client_negotiation.cc:769] Sending connection context
0807 15:17:23.539076 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0807 15:17:23.539509 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":172,"threads_started":1}
I20260807 15:17:23.542574 18701 heartbeater.cc:347] Connected to a master server at 127.31.230.254:34251
I20260807 15:17:23.542860 18701 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.543550 18701 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
W20260807 15:17:23.545260 18701 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:34251 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:17:23.545388 18701 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.545593 18701 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:23.546474 18701 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.546639 18701 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
W20260807 15:17:23.547478 18701 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:17:23.690510 18705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:23.690902 18705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:23.690984 18705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:23.697081 18705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:23.697165 18705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:23.697256 18705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:17:23.705265 18705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.230.212:53789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:37845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=40913
--tserver_master_addrs=127.31.230.254:34251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:23.706802 18705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:23.708244 18705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:23.722286 18715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:23.722309 18714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:23.722777 18705 server_base.cc:1034] running on GCE node
W20260807 15:17:23.722956 18718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:23.723412 18705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:23.724162 18705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:23.725386 18705 hybrid_clock.cc:648] HybridClock initialized: now 1786115843725361 us; error 51 us; skew 500 ppm
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:23.728272 18705 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:17:23.729278 18705 webserver.cc:460] Webserver started at http://127.31.230.195:40913/ using document root <none> and password file <none>
I20260807 15:17:23.730031 18705 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:23.730199 18705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:23.754617 18705 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 15:17:23.758209 18726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.759815 18705 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:17:23.759980 18705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "6cbcf961216c41a68ad9749d5dd265be"
format_stamp: "Formatted at 2026-08-07 15:17:22 on dist-test-slave-gkw7"
server_key: "7f473151365ccc244e74a2237dbf65b3"
server_key_iv: "49c7da6b9413120e803057bb6a024e92"
server_key_version: "kuduclusterkey@0"
I20260807 15:17:23.760600 18705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:23.784360 18705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:23.788166 18705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:23.788389 18705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:23.789148 18705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:23.790398 18705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:23.790481 18705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.790552 18705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:23.790598 18705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:23.802597 18705 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:37845
I20260807 15:17:23.802662 18895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:37845 every 8 connection(s)
I20260807 15:17:23.804054 18705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:17:23.810148 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 18705
Aug 07 15:17:23 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:23.822214 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.806817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:46783 (local address 127.31.230.254:34251)
0807 15:17:23.806954 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:23.806959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:23.807946 (+ 987us) server_negotiation.cc:407] Connection header received
0807 15:17:23.809206 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:23.809210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:23.809268 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:23.809323 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:23.811202 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.812054 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.813852 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.814231 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.814320 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.818111 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:23.818115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:23.818118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:23.818143 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:23.819655 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.820266 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.820269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.820271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.820352 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.820802 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.820807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.820810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.821054 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:23.821153 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:23.821995 (+ 842us) server_negotiation.cc:299] Negotiation successful
0807 15:17:23.822092 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:17:23.823761 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.807293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34251 (local address 127.31.230.195:46783)
0807 15:17:23.807786 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:23.807839 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:23.808947 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:23.809781 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:23.809789 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:23.810154 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:23.810980 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:23.811003 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.812225 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.812228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:23.813683 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:23.813695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.814498 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.814500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:23.814654 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.815461 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:23.815488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:23.817917 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:23.819784 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:23.819790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:23.819808 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:23.820170 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:23.820458 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:23.820461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:23.820463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:23.820670 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:23.821243 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:23.821252 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:23.821654 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 15:17:23.822447 (+ 793us) client_negotiation.cc:240] Negotiation successful
0807 15:17:23.822781 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":143,"threads_started":1}
I20260807 15:17:23.825681 18897 heartbeater.cc:347] Connected to a master server at 127.31.230.254:34251
I20260807 15:17:23.825958 18897 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.826514 18897 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
W20260807 15:17:23.828225 18897 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:34251 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:17:23.828364 18897 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.828540 18897 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:23.829433 18897 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:23.829649 18897 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
W20260807 15:17:23.830523 18897 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:17:23.832955 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:23.813664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.31.230.254:34251)
0807 15:17:23.815669 (+ 2005us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:23.815680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:23.815693 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:17:23.815743 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:23.815750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:23.815799 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:23.815885 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:23.816819 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.817436 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.818978 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:23.819991 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:23.820092 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:23.824330 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:23.824334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:23.824337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:23.824364 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:23.826078 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.831674 (+ 5596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.831679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.831680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.831771 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:23.832078 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:23.832081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:23.832083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:23.832272 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:23.832484 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:23.832646 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:17:23.832769 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1908,"thread_start_us":122,"threads_started":1}
Aug 07 15:17:24 dist-test-slave-gkw7 krb5kdc[16832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115844, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:24.237882 18512 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:24.238214 18512 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:24.239526 18227 ts_manager.cc:194] Registered new tserver with Master: 86637fbf953e46ab9f95c87639fe837e (127.31.230.193:32899)
I20260807 15:17:24.241236 18227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:51665
I20260807 15:17:24.547755 18701 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:24.548120 18701 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:24.549139 18227 ts_manager.cc:194] Registered new tserver with Master: b4b26409153b494cba87130cdbb14424 (127.31.230.194:45973)
I20260807 15:17:24.550011 18227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:59057
Aug 07 15:17:24 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115844, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:24.830765 18897 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:24.831136 18897 heartbeater.cc:511] Master 127.31.230.254:34251 requested a full tablet report, sending...
I20260807 15:17:24.832052 18227 ts_manager.cc:194] Registered new tserver with Master: 6cbcf961216c41a68ad9749d5dd265be (127.31.230.195:37845)
I20260807 15:17:24.832952 18227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:46783
I20260807 15:17:24.839232 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:17:24.879805 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.841343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.31.230.254:34251)
0807 15:17:24.849828 (+ 8485us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:24.849835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:24.850294 (+ 459us) server_negotiation.cc:407] Connection header received
0807 15:17:24.850356 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:24.850359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:24.850416 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:24.850501 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:24.851624 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.852640 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.854088 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.854622 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.854748 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.861548 (+ 6800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:24.861555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:24.861561 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:24.861599 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:24.875137 (+ 13538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:24.875861 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:24.875865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:24.875867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:24.875983 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:24.878547 (+ 2564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:24.878551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:24.878552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:24.878841 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:24.878996 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:24.879439 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 15:17:24.879603 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8383,"thread_start_us":83,"threads_started":1}
I20260807 15:17:24.894415 18227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:17:24.899761 18227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:17:24.956864 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.950478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:32899 (local address 127.0.0.1:34154)
0807 15:17:24.950929 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:24.950988 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:24.951136 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:24.951896 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:24.951903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:24.951928 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:24.952088 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:24.952095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.953689 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.953693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:24.955178 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:24.955188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.956339 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.956343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:24.956461 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.956504 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:17:24.956587 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:17:24.956717 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":139,"threads_started":1}
I20260807 15:17:24.958146 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.950596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.31.230.193:32899)
0807 15:17:24.951421 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:24.951428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:24.951452 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:17:24.951527 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:24.951532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:24.951731 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:24.952268 (+ 537us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:24.952400 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.953554 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.955330 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.956171 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.957720 (+ 1549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.957858 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:24.957905 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 15:17:24.957993 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":107,"threads_started":1}
I20260807 15:17:24.961796 18418 tablet_service.cc:1467] Processing CreateTablet for tablet 3b41aebe9a5f4baf984c4f3ada337e8e (DEFAULT_TABLE table=test-table [id=29f69697b439455297cd89b69ba4af1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:24.963239 18418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b41aebe9a5f4baf984c4f3ada337e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:24.968647 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.951450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37845 (local address 127.0.0.1:37676)
0807 15:17:24.957475 (+ 6025us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:24.957489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:24.957576 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:24.964487 (+ 6911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:24.964490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:24.964510 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:24.964586 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:24.964592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.966040 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.966044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:24.967145 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:24.967157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.968253 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.968257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:24.968347 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.968375 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:17:24.968427 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:17:24.968500 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5969,"thread_start_us":115,"threads_started":1}
I20260807 15:17:24.969419 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.951863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.31.230.195:37845)
0807 15:17:24.958797 (+ 6934us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:24.958805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:24.958837 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:17:24.958921 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:24.958926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:24.959143 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:24.959292 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:24.964740 (+ 5448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.965905 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.967315 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.968096 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.968871 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.969005 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:24.969057 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 15:17:24.969159 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6085,"thread_start_us":93,"threads_started":1}
I20260807 15:17:24.972719 18768 tablet_service.cc:1467] Processing CreateTablet for tablet 3b41aebe9a5f4baf984c4f3ada337e8e (DEFAULT_TABLE table=test-table [id=29f69697b439455297cd89b69ba4af1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:24.973922 18768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b41aebe9a5f4baf984c4f3ada337e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:24.986785 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.953244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:45973 (local address 127.0.0.1:52662)
0807 15:17:24.957706 (+ 4462us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:24.957722 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:24.957812 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:24.963411 (+ 5599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:24.963415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:24.963435 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:24.963530 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:24.963536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.967586 (+ 4050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.967590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:24.968709 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:24.968721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.986253 (+ 17532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.986258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:24.986380 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.986411 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 15:17:24.986508 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:17:24.986619 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4394,"thread_start_us":92,"threads_started":1}
I20260807 15:17:24.987471 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:24.953408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.31.230.194:45973)
0807 15:17:24.962770 (+ 9362us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:24.962777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:24.962802 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:17:24.962879 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:24.962884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:24.963086 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:24.963234 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:24.963685 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.967443 (+ 3758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.968985 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:24.986084 (+ 17099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:24.986251 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:24.986390 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:24.987217 (+ 827us) server_negotiation.cc:299] Negotiation successful
0807 15:17:24.987353 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9249,"thread_start_us":111,"threads_started":1}
I20260807 15:17:24.987586 18954 tablet_bootstrap.cc:492] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e: Bootstrap starting.
I20260807 15:17:24.990530 18954 tablet_bootstrap.cc:654] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:24.990762 18610 tablet_service.cc:1467] Processing CreateTablet for tablet 3b41aebe9a5f4baf984c4f3ada337e8e (DEFAULT_TABLE table=test-table [id=29f69697b439455297cd89b69ba4af1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:24.991528 18954 log.cc:824] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:24.992086 18610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b41aebe9a5f4baf984c4f3ada337e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:25.002053 18955 tablet_bootstrap.cc:492] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Bootstrap starting.
I20260807 15:17:25.005955 18954 tablet_bootstrap.cc:492] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e: No bootstrap required, opened a new log
I20260807 15:17:25.006229 18954 ts_tablet_manager.cc:1397] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.002s
I20260807 15:17:25.007215 18955 tablet_bootstrap.cc:654] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:25.008158 18955 log.cc:824] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:25.010007 18957 tablet_bootstrap.cc:492] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Bootstrap starting.
I20260807 15:17:25.010686 18954 raft_consensus.cc:348] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.011132 18954 raft_consensus.cc:374] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:25.011237 18954 raft_consensus.cc:729] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86637fbf953e46ab9f95c87639fe837e, State: Initialized, Role: FOLLOWER
I20260807 15:17:25.011842 18954 consensus_queue.cc:260] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.012557 18957 tablet_bootstrap.cc:654] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:25.013180 18954 ts_tablet_manager.cc:1428] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260807 15:17:25.013478 18957 log.cc:824] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:25.013676 18512 heartbeater.cc:503] Master 127.31.230.254:34251 was elected leader, sending a full tablet report...
I20260807 15:17:25.019977 18957 tablet_bootstrap.cc:492] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: No bootstrap required, opened a new log
I20260807 15:17:25.020203 18955 tablet_bootstrap.cc:492] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: No bootstrap required, opened a new log
I20260807 15:17:25.020282 18957 ts_tablet_manager.cc:1397] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 15:17:25.020548 18955 ts_tablet_manager.cc:1397] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.007s
I20260807 15:17:25.026322 18957 raft_consensus.cc:348] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.026680 18957 raft_consensus.cc:374] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:25.026772 18957 raft_consensus.cc:729] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b26409153b494cba87130cdbb14424, State: Initialized, Role: FOLLOWER
I20260807 15:17:25.026758 18955 raft_consensus.cc:348] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.027072 18955 raft_consensus.cc:374] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:25.027158 18955 raft_consensus.cc:729] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbcf961216c41a68ad9749d5dd265be, State: Initialized, Role: FOLLOWER
Aug 07 15:17:25 dist-test-slave-gkw7 krb5kdc[16832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115844, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for HTTP/127.31.230.212@KRBTEST.COM
I20260807 15:17:25.027374 18957 consensus_queue.cc:260] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.027673 18955 consensus_queue.cc:260] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.028594 18957 ts_tablet_manager.cc:1428] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 15:17:25.028823 18955 ts_tablet_manager.cc:1428] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 15:17:25.028980 18701 heartbeater.cc:503] Master 127.31.230.254:34251 was elected leader, sending a full tablet report...
I20260807 15:17:25.029215 18897 heartbeater.cc:503] Master 127.31.230.254:34251 was elected leader, sending a full tablet report...
I20260807 15:17:25.043531 18963 raft_consensus.cc:482] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:17:25.043862 18963 raft_consensus.cc:504] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.046334 18963 leader_election.cc:288] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cbcf961216c41a68ad9749d5dd265be (127.31.230.195:37845), 86637fbf953e46ab9f95c87639fe837e (127.31.230.193:32899)
I20260807 15:17:25.051558 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:25.045767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37845 (local address 127.31.230.194:59315)
0807 15:17:25.046861 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:25.046877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:25.047055 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:25.047472 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:25.047475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:25.047492 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:25.047658 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:25.047667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.048672 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.048675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:25.050041 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:25.050053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.051066 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.051069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:25.051187 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:25.051214 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:17:25.051294 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:17:25.051372 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"thread_start_us":139,"threads_started":1}
I20260807 15:17:25.051566 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:25.046285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59315 (local address 127.31.230.195:37845)
0807 15:17:25.046436 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:25.046445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:25.046956 (+ 511us) server_negotiation.cc:407] Connection header received
0807 15:17:25.047207 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:25.047211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:25.047260 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:25.047315 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:25.047797 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.048512 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.050197 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.050809 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.050910 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:25.050965 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:25.051343 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 15:17:25.051437 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:17:25.053313 18825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b41aebe9a5f4baf984c4f3ada337e8e" candidate_uuid: "b4b26409153b494cba87130cdbb14424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbcf961216c41a68ad9749d5dd265be" is_pre_election: true
I20260807 15:17:25.053722 18825 raft_consensus.cc:2454] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4b26409153b494cba87130cdbb14424 in term 0.
I20260807 15:17:25.054272 18542 leader_election.cc:302] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cbcf961216c41a68ad9749d5dd265be, b4b26409153b494cba87130cdbb14424; no voters:
I20260807 15:17:25.054679 18963 raft_consensus.cc:2781] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:17:25.054780 18963 raft_consensus.cc:482] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:17:25.054829 18963 raft_consensus.cc:3037] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:25.056197 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:25.047008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:43963 (local address 127.31.230.193:32899)
0807 15:17:25.047413 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:25.047419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:25.047434 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:17:25.047484 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:25.047487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:25.047534 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:25.047600 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:25.049841 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.050591 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.054826 (+ 4235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.055431 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.055536 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:25.055598 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:25.055962 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:17:25.056065 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260807 15:17:25.056450 18963 raft_consensus.cc:504] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.056838 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:25.046552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:32899 (local address 127.31.230.194:43963)
0807 15:17:25.047128 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:25.047143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:25.047246 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:25.049464 (+ 2218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:25.049466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:25.049486 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:25.049659 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:25.049668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.050778 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.050781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:25.054618 (+ 3837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:25.054632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.055711 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.055714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:25.055812 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:25.055834 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:17:25.056621 (+ 787us) client_negotiation.cc:240] Negotiation successful
0807 15:17:25.056700 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":120,"threads_started":1}
I20260807 15:17:25.057261 18825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b41aebe9a5f4baf984c4f3ada337e8e" candidate_uuid: "b4b26409153b494cba87130cdbb14424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbcf961216c41a68ad9749d5dd265be"
I20260807 15:17:25.057395 18825 raft_consensus.cc:3037] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 0 FOLLOWER]: Advancing to term 1
W20260807 15:17:25.057600 18901 tablet.cc:2369] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:17:25.058770 18825 raft_consensus.cc:2454] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b26409153b494cba87130cdbb14424 in term 1.
I20260807 15:17:25.056897 18963 leader_election.cc:288] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [CANDIDATE]: Term 1 election: Requested vote from peers 6cbcf961216c41a68ad9749d5dd265be (127.31.230.195:37845), 86637fbf953e46ab9f95c87639fe837e (127.31.230.193:32899)
I20260807 15:17:25.059232 18542 leader_election.cc:302] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cbcf961216c41a68ad9749d5dd265be, b4b26409153b494cba87130cdbb14424; no voters:
I20260807 15:17:25.059700 18963 raft_consensus.cc:2781] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:25.059834 18963 raft_consensus.cc:686] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 1 LEADER]: Becoming Leader. State: Replica: b4b26409153b494cba87130cdbb14424, State: Running, Role: LEADER
I20260807 15:17:25.060279 18963 consensus_queue.cc:237] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } }
I20260807 15:17:25.063292 18438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b41aebe9a5f4baf984c4f3ada337e8e" candidate_uuid: "b4b26409153b494cba87130cdbb14424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86637fbf953e46ab9f95c87639fe837e"
I20260807 15:17:25.063577 18438 raft_consensus.cc:3037] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:25.064313 18226 catalog_manager.cc:5495] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4b26409153b494cba87130cdbb14424 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "b4b26409153b494cba87130cdbb14424" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4b26409153b494cba87130cdbb14424" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 45973 } health_report { overall_health: HEALTHY } } }
I20260807 15:17:25.065085 18438 raft_consensus.cc:2454] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b26409153b494cba87130cdbb14424 in term 1.
I20260807 15:17:25.063292 18440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b41aebe9a5f4baf984c4f3ada337e8e" candidate_uuid: "b4b26409153b494cba87130cdbb14424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86637fbf953e46ab9f95c87639fe837e" is_pre_election: true
I20260807 15:17:25.065795 18440 raft_consensus.cc:2362] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4b26409153b494cba87130cdbb14424 in term 1. Re-sending same reply.
I20260807 15:17:25.089824 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:25.084828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.31.230.194:45973)
0807 15:17:25.084951 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:25.084956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:25.084970 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:17:25.085061 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:25.085063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:25.085119 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:25.085253 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:17:25.085512 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.086327 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.087582 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:25.087906 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:25.087998 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:25.088424 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:17:25.089004 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:17:25.089143 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:25.089590 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 15:17:25.089657 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:17:25.097808 18440 raft_consensus.cc:1261] T 3b41aebe9a5f4baf984c4f3ada337e8e P 86637fbf953e46ab9f95c87639fe837e [term 1 FOLLOWER]: Refusing update from remote peer b4b26409153b494cba87130cdbb14424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:25.098713 18969 consensus_queue.cc:1035] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86637fbf953e46ab9f95c87639fe837e" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:25.100207 18825 raft_consensus.cc:1261] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 1 FOLLOWER]: Refusing update from remote peer b4b26409153b494cba87130cdbb14424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:25.103317 18969 consensus_queue.cc:1035] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cbcf961216c41a68ad9749d5dd265be" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:25.125636 18977 mvcc.cc:205] Tried to move back new op lower bound from 7315930501510397952 to 7315930501370937344. Current Snapshot: MvccSnapshot[applied={T|T < 7315930501510397952}]
I20260807 15:17:25.126534 18978 mvcc.cc:205] Tried to move back new op lower bound from 7315930501510397952 to 7315930501370937344. Current Snapshot: MvccSnapshot[applied={T|T < 7315930501510397952}]
I20260807 15:17:25.150674 18227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:25.150887 18227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:25.153770 18227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87e4457530e44045b87bc4cd20c389be: Sending DeleteTablet for 3 replicas of tablet 3b41aebe9a5f4baf984c4f3ada337e8e
I20260807 15:17:25.154748 18768 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b41aebe9a5f4baf984c4f3ada337e8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:25 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 15:17:25.154879 18418 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b41aebe9a5f4baf984c4f3ada337e8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:25 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34154
I20260807 15:17:25.155858 18610 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b41aebe9a5f4baf984c4f3ada337e8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:25 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:52662
I20260807 15:17:25.157003 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 18282
I20260807 15:17:25.168859 18990 tablet_replica.cc:330] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: stopping tablet replica
I20260807 15:17:25.169368 18990 raft_consensus.cc:2229] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:17:25.174193 18993 tablet_replica.cc:330] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: stopping tablet replica
I20260807 15:17:25.174707 18990 raft_consensus.cc:2258] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:25.175153 18993 raft_consensus.cc:2229] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:17:25.175633 18993 raft_consensus.cc:2258] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:25.177518 18990 ts_tablet_manager.cc:1905] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:25.178938 18993 ts_tablet_manager.cc:1905] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:25.184129 18993 ts_tablet_manager.cc:1918] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:25.186241 18990 ts_tablet_manager.cc:1918] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:25.186419 18993 log.cc:1196] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/3b41aebe9a5f4baf984c4f3ada337e8e
I20260807 15:17:25.186836 18993 ts_tablet_manager.cc:1939] T 3b41aebe9a5f4baf984c4f3ada337e8e P b4b26409153b494cba87130cdbb14424: Deleting consensus metadata
I20260807 15:17:25.187008 18990 log.cc:1196] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/3b41aebe9a5f4baf984c4f3ada337e8e
I20260807 15:17:25.187431 18990 ts_tablet_manager.cc:1939] T 3b41aebe9a5f4baf984c4f3ada337e8e P 6cbcf961216c41a68ad9749d5dd265be: Deleting consensus metadata
I20260807 15:17:25.188876 18206 catalog_manager.cc:4841] TS b4b26409153b494cba87130cdbb14424 (127.31.230.194:45973): tablet 3b41aebe9a5f4baf984c4f3ada337e8e (table test-table [id=29f69697b439455297cd89b69ba4af1a]) successfully deleted
I20260807 15:17:25.192319 18205 catalog_manager.cc:4841] TS 6cbcf961216c41a68ad9749d5dd265be (127.31.230.195:37845): tablet 3b41aebe9a5f4baf984c4f3ada337e8e (table test-table [id=29f69697b439455297cd89b69ba4af1a]) successfully deleted
W20260807 15:17:25.192984 18206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:32899 (error 108)
I20260807 15:17:25.194684 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 18518
I20260807 15:17:25.194710 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:25.194400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:32899 (local address 127.0.0.1:34158)
0807 15:17:25.194521 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:25.194614 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:32899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:17:25.197052 18206 catalog_manager.cc:4584] TS 86637fbf953e46ab9f95c87639fe837e (127.31.230.193:32899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b41aebe9a5f4baf984c4f3ada337e8e: Network error: Client connection negotiation failed: client connection to 127.31.230.193:32899: connect: Connection refused (error 111)
I20260807 15:17:25.213342 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 18705
I20260807 15:17:25.229321 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 18181
I20260807 15:17:25.248232 32667 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:17:25.598842 32667 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:17:25.598984 32667 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:17:25.944667 32667 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:17:25.944 UTC [16870] LOG: received smart shutdown request
2026-08-07 15:17:25.949 UTC [16870] LOG: background worker "logical replication launcher" (PID 16882) exited with exit code 1
2026-08-07 15:17:25.949 UTC [16876] LOG: shutting down
2026-08-07 15:17:26.122 UTC [16870] LOG: database system is shut down
2026-08-07T15:17:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49599 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:17:26 dist-test-slave-gkw7 krb5kdc[19010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:17:26 dist-test-slave-gkw7 krb5kdc[19010](info): set up 2 sockets
Aug 07 15:17:26 dist-test-slave-gkw7 krb5kdc[19010](info): commencing operation
krb5kdc: starting...
W20260807 15:17:27.525789 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:17:27 dist-test-slave-gkw7 krb5kdc[19010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:17:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:17:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:17:27.661278 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34221
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:34221,127.31.230.253:41725,127.31.230.252:34077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:27.779632 19029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:27.779959 19029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:27.780043 19029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:27.784159 19029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:27.784248 19029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:27.784287 19029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:27.784343 19029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:27.784381 19029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:27.789746 19029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:34221,127.31.230.253:41725,127.31.230.252:34077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:34221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:27.790849 19029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:27.791908 19029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:27.800998 19037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:27.801009 19042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:27.800999 19039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:27.801433 19029 server_base.cc:1034] running on GCE node
I20260807 15:17:27.802045 19029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:27.803194 19029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:27.804414 19029 hybrid_clock.cc:648] HybridClock initialized: now 1786115847804391 us; error 32 us; skew 500 ppm
Aug 07 15:17:27 dist-test-slave-gkw7 krb5kdc[19010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:27.807466 19029 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:27.808372 19029 webserver.cc:460] Webserver started at http://127.31.230.254:43059/ using document root <none> and password file <none>
I20260807 15:17:27.808974 19029 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:27.809041 19029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:27.809300 19029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:27.811652 19029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "b91460625fe846e5a01fcea51f5130f8"
format_stamp: "Formatted at 2026-08-07 15:17:27 on dist-test-slave-gkw7"
server_key: "c27e6ede27cb0a6271686d07236cd0ac"
server_key_iv: "7d0d11a9b2d040589a041fd2aaf9fb92"
server_key_version: "encryptionkey@0"
I20260807 15:17:27.812282 19029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "b91460625fe846e5a01fcea51f5130f8"
format_stamp: "Formatted at 2026-08-07 15:17:27 on dist-test-slave-gkw7"
server_key: "c27e6ede27cb0a6271686d07236cd0ac"
server_key_iv: "7d0d11a9b2d040589a041fd2aaf9fb92"
server_key_version: "encryptionkey@0"
I20260807 15:17:27.816233 19029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:17:27.818899 19053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:27.820127 19029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:17:27.820276 19029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "b91460625fe846e5a01fcea51f5130f8"
format_stamp: "Formatted at 2026-08-07 15:17:27 on dist-test-slave-gkw7"
server_key: "c27e6ede27cb0a6271686d07236cd0ac"
server_key_iv: "7d0d11a9b2d040589a041fd2aaf9fb92"
server_key_version: "encryptionkey@0"
I20260807 15:17:27.820382 19029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:27.837069 19029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:27.840093 19029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:27.840310 19029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:27.847648 19029 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:34221
I20260807 15:17:27.847944 19126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:34221 every 8 connection(s)
I20260807 15:17:27.848641 19029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:27.849292 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19029
I20260807 15:17:27.849452 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:17:27.849782 32667 external_mini_cluster.cc:1428] Setting key e85444f40de120485b42472d0946fa86
I20260807 15:17:27.851584 19127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:27.855383 19127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.230.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.253@KRBTEST.COM" created.
Aug 07 15:17:27 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:27.870766 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.857639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34221 (local address 127.0.0.1:60082)
0807 15:17:27.858140 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.858195 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:27.859108 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:27.859797 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:27.859802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:27.860231 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:27.860476 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:27.860489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:27.861762 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:27.861765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:27.862500 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:27.862509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:27.863160 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:27.863164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:27.863332 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:27.864022 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:27.864044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:27.865614 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:27.867597 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:27.867607 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:27.867619 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:27.867855 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:27.868190 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:27.868193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:27.868195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:27.868310 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:27.868915 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:27.868921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:27.869124 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 15:17:27.869317 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 15:17:27.869743 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":216,"spinlock_wait_cycles":1280,"thread_start_us":145,"threads_started":1}
I20260807 15:17:27.870766 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.858529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.31.230.254:34221)
0807 15:17:27.858833 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:27.858843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:27.858880 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:17:27.859424 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:27.859429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:27.859614 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:27.859811 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:27.860626 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:27.861628 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:27.862633 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:27.863010 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:27.863192 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:27.865767 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:27.865773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:27.865782 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:27.865803 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:27.867480 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:27.867970 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:27.867983 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:27.867984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:27.868087 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:27.868402 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:27.868407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:27.868409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:27.868797 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:27.868944 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:27.869392 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 15:17:27.869691 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
I20260807 15:17:27.873396 19127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:17:27.874209 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.873932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44846)
0807 15:17:27.874061 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.874143 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 15:17:27.874403 19057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
I20260807 15:17:27.875234 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.875005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44852)
0807 15:17:27.875128 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.875176 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 15:17:27.875370 19127 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:41725: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab.
Entry for principal kudu/127.31.230.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab.
I20260807 15:17:27.878818 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:41725
--webserver_interface=127.31.230.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab
--principal=kudu/127.31.230.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:34221,127.31.230.253:41725,127.31.230.252:34077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:27.924651 19127 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } attempt: 1
I20260807 15:17:27.925674 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.925320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44868)
0807 15:17:27.925466 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.925583 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 15:17:27.926595 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.926288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44882)
0807 15:17:27.926444 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.926506 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 15:17:27.926781 19127 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:41725: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
I20260807 15:17:27.995010 19127 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } attempt: 2
W20260807 15:17:27.995232 19136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:27.995674 19136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:27.995746 19136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 15:17:27.996101 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.995774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44888)
0807 15:17:27.995942 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.996011 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:17:27.997007 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:27.996737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44894)
0807 15:17:27.996880 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:27.996942 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 15:17:27.997263 19127 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:41725: Network error: Client connection negotiation failed: client connection to 127.31.230.253:41725: connect: Connection refused (error 111)
W20260807 15:17:28.000360 19136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:28.000452 19136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:28.000500 19136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:28.000559 19136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:28.000598 19136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:28.006224 19136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:34221,127.31.230.253:41725,127.31.230.252:34077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab
--principal=kudu/127.31.230.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:41725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.230.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:28.007462 19136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:28.008594 19136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:28.018349 19144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:28.018350 19146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:28.018415 19136 server_base.cc:1034] running on GCE node
W20260807 15:17:28.018349 19143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:28.019157 19136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:28.020340 19136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:28.021631 19136 hybrid_clock.cc:648] HybridClock initialized: now 1786115848021614 us; error 25 us; skew 500 ppm
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:28.025149 19136 init.cc:377] Logged in from keytab as kudu/127.31.230.253@KRBTEST.COM (short username kudu)
I20260807 15:17:28.026301 19136 webserver.cc:460] Webserver started at http://127.31.230.253:36853/ using document root <none> and password file <none>
I20260807 15:17:28.026924 19136 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:28.027027 19136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:28.027343 19136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:28.029708 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data/instance:
uuid: "b6d3d70b47f04f6796746556a42ac08d"
format_stamp: "Formatted at 2026-08-07 15:17:28 on dist-test-slave-gkw7"
server_key: "359eef2f4cae8314b38f136fabc13d93"
server_key_iv: "2f10aaa54375095596ee7b3b8be59abe"
server_key_version: "encryptionkey@0"
I20260807 15:17:28.030365 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal/instance:
uuid: "b6d3d70b47f04f6796746556a42ac08d"
format_stamp: "Formatted at 2026-08-07 15:17:28 on dist-test-slave-gkw7"
server_key: "359eef2f4cae8314b38f136fabc13d93"
server_key_iv: "2f10aaa54375095596ee7b3b8be59abe"
server_key_version: "encryptionkey@0"
I20260807 15:17:28.034617 19136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:17:28.037295 19159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:28.038658 19136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:17:28.038812 19136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal
uuid: "b6d3d70b47f04f6796746556a42ac08d"
format_stamp: "Formatted at 2026-08-07 15:17:28 on dist-test-slave-gkw7"
server_key: "359eef2f4cae8314b38f136fabc13d93"
server_key_iv: "2f10aaa54375095596ee7b3b8be59abe"
server_key_version: "encryptionkey@0"
I20260807 15:17:28.038928 19136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:28.049190 19136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:28.052484 19136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:28.052708 19136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:28.060281 19136 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.253:41725
I20260807 15:17:28.060581 19231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.253:41725 every 8 connection(s)
I20260807 15:17:28.061357 19136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
I20260807 15:17:28.064455 19232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:28.066890 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19136
I20260807 15:17:28.067039 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-1/wal/instance
I20260807 15:17:28.067375 32667 external_mini_cluster.cc:1428] Setting key 1fb4c5056684a93e99a5394581eb17b9
I20260807 15:17:28.069571 19232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.230.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.252@KRBTEST.COM" created.
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:28.085851 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.071650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.31.230.254:34221)
0807 15:17:28.071841 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.071848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.072730 (+ 882us) server_negotiation.cc:407] Connection header received
0807 15:17:28.073751 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.073758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.073832 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.073955 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.075200 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.076093 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.077376 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.077716 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.077832 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.080772 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.080776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.080779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.080811 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.083347 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.083886 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.083893 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.083898 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.083999 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.084372 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.084378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.084383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.084682 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.084803 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.085437 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.085618 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:17:28.086479 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.072027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34221 (local address 127.0.0.1:60094)
0807 15:17:28.072552 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.072606 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.073475 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.074113 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.074119 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.074538 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.075035 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.075048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.076279 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.076282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.077257 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.077267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.077979 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.077982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.078132 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.078839 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.078857 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.080589 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.083479 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.083485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.083500 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.083774 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.084101 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.084103 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.084105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.084269 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.084797 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.084802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.085130 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.085333 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.085677 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":164,"threads_started":1}
I20260807 15:17:28.087978 19232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:17:28.088441 19127 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } attempt: 3
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:17:28.099671 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.088838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44906)
0807 15:17:28.088959 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.088973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.089049 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.089882 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.089885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.090067 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.090177 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.090183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.091385 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.091390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.092073 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.092083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.092801 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.092806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.092921 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.093641 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.093662 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.095080 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.097800 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.097802 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.097804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.097998 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.098337 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.098339 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.098340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.098455 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.099003 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.099006 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.099196 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.099299 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.099422 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:17:28.099794 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.089200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44906 (local address 127.31.230.253:41725)
0807 15:17:28.089571 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.089581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.089596 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:17:28.089633 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.089636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.089761 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.089902 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.090302 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.091262 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.092201 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.092670 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.092813 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.095293 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.095306 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.095333 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.095366 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.097698 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.098132 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.098142 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.098144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.098238 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.098552 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.098554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.098556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.098899 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.099032 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.099423 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.099588 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":257,"spinlock_wait_cycles":10496,"thread_start_us":101,"threads_started":1}
I20260807 15:17:28.100028 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.088527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44902)
0807 15:17:28.088767 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.088785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.088866 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.089873 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.089875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.090054 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.090169 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.090175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.091418 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.091421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.092165 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.092175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.092813 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.092816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.092925 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.093618 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.093633 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.095661 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.098456 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.098459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.098462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.098718 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.099002 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.099005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.099007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.099105 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.099505 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.099508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.099600 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.099703 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.099877 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":2}
I20260807 15:17:28.100262 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.088636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.31.230.253:41725)
0807 15:17:28.089177 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.089197 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.089220 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:17:28.089474 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.089481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.089751 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.089901 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.090280 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.091303 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.092284 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.092691 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.092856 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.095905 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.095909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.095912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.095941 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.098345 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.098796 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.098799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.098801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.098880 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.099187 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.099189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.099190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.099393 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.099487 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.099779 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.100026 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":517,"server-negotiator.queue_time_us":440,"spinlock_wait_cycles":13696,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab.
Entry for principal kudu/127.31.230.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab.
I20260807 15:17:28.102207 19127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:17:28.102339 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:34077
--webserver_interface=127.31.230.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab
--principal=kudu/127.31.230.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:34221,127.31.230.253:41725,127.31.230.252:34077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:28.102962 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.102707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47816)
0807 15:17:28.102838 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.102891 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 15:17:28.103727 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.103482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47820)
0807 15:17:28.103621 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.103674 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 15:17:28.103926 19127 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:34077: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
I20260807 15:17:28.104676 19232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:17:28.105456 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.105132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47822)
0807 15:17:28.105275 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.105369 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 15:17:28.105710 19162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
I20260807 15:17:28.106683 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.106398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47830)
0807 15:17:28.106541 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.106603 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:17:28.106837 19232 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:34077: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
I20260807 15:17:28.135107 19127 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } attempt: 1
I20260807 15:17:28.136085 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.135762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47844)
0807 15:17:28.135925 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.135999 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 15:17:28.136916 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.136695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47850)
0807 15:17:28.136806 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.136855 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:17:28.137110 19127 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:34077: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
I20260807 15:17:28.156219 19232 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } attempt: 1
I20260807 15:17:28.157271 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.156885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47860)
0807 15:17:28.157077 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.157162 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 15:17:28.158100 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.157861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47870)
0807 15:17:28.157983 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.158036 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 15:17:28.158275 19232 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:34077: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
I20260807 15:17:28.212286 19127 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } attempt: 2
I20260807 15:17:28.213176 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.212860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47884)
0807 15:17:28.213020 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.213091 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 15:17:28.214004 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.213794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47886)
0807 15:17:28.213901 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.213954 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 15:17:28.214207 19127 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:34077: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
W20260807 15:17:28.221380 19240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:28.221698 19240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:28.221745 19240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:28.225873 19240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:28.225925 19240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:28.225945 19240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:28.225962 19240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:28.225987 19240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:28.226444 19232 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } attempt: 2
I20260807 15:17:28.227267 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.226920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47900)
0807 15:17:28.227089 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.227167 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 15:17:28.227981 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.227759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47916)
0807 15:17:28.227869 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.227920 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 15:17:28.228152 19232 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:34077: Network error: Client connection negotiation failed: client connection to 127.31.230.252:34077: connect: Connection refused (error 111)
I20260807 15:17:28.231761 19240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:34221,127.31.230.253:41725,127.31.230.252:34077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab
--principal=kudu/127.31.230.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:34077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.230.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:28.232882 19240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:28.234037 19240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:28.243111 19253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:28.243124 19251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:28.243322 19240 server_base.cc:1034] running on GCE node
W20260807 15:17:28.243405 19250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:28.243958 19240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:28.245132 19240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:28.246354 19240 hybrid_clock.cc:648] HybridClock initialized: now 1786115848246321 us; error 44 us; skew 500 ppm
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:28.249131 19240 init.cc:377] Logged in from keytab as kudu/127.31.230.252@KRBTEST.COM (short username kudu)
I20260807 15:17:28.250154 19240 webserver.cc:460] Webserver started at http://127.31.230.252:39971/ using document root <none> and password file <none>
I20260807 15:17:28.250744 19240 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:28.250837 19240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:28.251075 19240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:28.253139 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data/instance:
uuid: "2085d6b3b0a6466e9617f00d72a80669"
format_stamp: "Formatted at 2026-08-07 15:17:28 on dist-test-slave-gkw7"
server_key: "ea7f44ba1959db93c4179bd974ac6dca"
server_key_iv: "8778a9472f2aa5a91dc05ba56171d76d"
server_key_version: "encryptionkey@0"
I20260807 15:17:28.253772 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal/instance:
uuid: "2085d6b3b0a6466e9617f00d72a80669"
format_stamp: "Formatted at 2026-08-07 15:17:28 on dist-test-slave-gkw7"
server_key: "ea7f44ba1959db93c4179bd974ac6dca"
server_key_iv: "8778a9472f2aa5a91dc05ba56171d76d"
server_key_version: "encryptionkey@0"
I20260807 15:17:28.257648 19240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:17:28.260160 19261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:28.261436 19240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:17:28.261631 19240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal
uuid: "2085d6b3b0a6466e9617f00d72a80669"
format_stamp: "Formatted at 2026-08-07 15:17:28 on dist-test-slave-gkw7"
server_key: "ea7f44ba1959db93c4179bd974ac6dca"
server_key_iv: "8778a9472f2aa5a91dc05ba56171d76d"
server_key_version: "encryptionkey@0"
I20260807 15:17:28.261754 19240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:28.273959 19240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:28.276832 19240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:28.277055 19240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:28.284432 19240 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.252:34077
I20260807 15:17:28.284713 19328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.252:34077 every 8 connection(s)
I20260807 15:17:28.285445 19240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
I20260807 15:17:28.288640 19329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:28.291138 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19240
I20260807 15:17:28.291293 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115680745020-32667-0/minicluster-data/master-2/wal/instance
I20260807 15:17:28.291637 32667 external_mini_cluster.cc:1428] Setting key c0556e903373f1b9ee3db1f35e8647e0
I20260807 15:17:28.292894 19329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:17:28.305840 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.294765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.31.230.254:34221)
0807 15:17:28.294922 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.294926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.295229 (+ 303us) server_negotiation.cc:407] Connection header received
0807 15:17:28.295305 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.295310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.295369 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.295460 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.296493 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.297284 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.298708 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.299060 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.299221 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.302054 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.302060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.302062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.302093 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.303544 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.304062 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.304067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.304068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.304190 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.304557 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.304560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.304562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.304779 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.304893 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.305488 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.305661 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:17:28.306744 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.293174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.31.230.252:34077)
0807 15:17:28.293692 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.293716 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.293755 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:17:28.294467 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.294485 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.294945 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.296200 (+ 1255us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.296367 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.297638 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.298796 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.299359 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.299447 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.302656 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.302662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.302673 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.302694 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.304412 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.304973 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.304977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.304978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.305057 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.305407 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.305410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.305413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.305734 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.305948 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.306214 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.306354 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":320,"spinlock_wait_cycles":896,"thread_start_us":155,"threads_started":1}
I20260807 15:17:28.306752 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.294666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:34221 (local address 127.0.0.1:60102)
0807 15:17:28.295025 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.295075 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.295212 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.295512 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.295519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.295845 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.296346 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.296359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.297393 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.297395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.298576 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.298588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.299266 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.299269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.299394 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.300071 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.300098 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.301898 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.303662 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.303673 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.303684 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.303955 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.304301 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.304304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.304308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.304446 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.304894 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.304902 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.305192 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.305423 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.305739 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260807 15:17:28.308010 19329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:17:28.313390 19127 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } attempt: 3
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:28.318027 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.308529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:41725 (local address 127.0.0.1:44920)
0807 15:17:28.308680 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.308694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.308764 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.309030 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.309032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.309215 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.309348 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.309356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.310291 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.310298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.311115 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.311125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.311677 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.311681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.311781 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.312261 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.312281 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.314115 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.316337 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.316343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.316345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.316573 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.316921 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.316923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.316924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.316997 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.317451 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.317456 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.317578 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.317694 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.317832 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 15:17:28.318094 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.308663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.31.230.253:41725)
0807 15:17:28.308800 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.308804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.308817 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:17:28.308854 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.308856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.308923 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.309010 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.309553 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.310167 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.311238 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.311563 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.311651 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.314349 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.314355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.314358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.314390 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.316234 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.316702 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.316709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.316712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.316807 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.317094 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.317099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.317103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.317350 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.317449 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.317779 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.317914 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:17:28.318816 19329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:17:28.319475 19232 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } attempt: 3
I20260807 15:17:28.321686 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.309503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.31.230.254:34221)
0807 15:17:28.309658 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.309663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.309716 (+ 53us) server_negotiation.cc:407] Connection header received
0807 15:17:28.309846 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.309849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.309894 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.309981 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.310744 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.311316 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.312371 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.312587 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.312670 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.315756 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.315763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.315765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.315790 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.317533 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.318036 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.318040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.318041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.318482 (+ 441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.318822 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.318827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.318829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.319129 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.319357 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.321032 (+ 1675us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.321491 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:17:28.324396 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.313831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47926)
0807 15:17:28.313993 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.314006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.314097 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.314382 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.314384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.314573 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.314698 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.314704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.315777 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.315780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.316609 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.316627 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.317272 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.317276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.317365 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.318284 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.318299 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.320739 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.322705 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.322708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.322710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.322959 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.323271 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.323274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.323276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.323358 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.323772 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.323776 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.323881 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.324044 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.324210 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 15:17:28.324433 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.313927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.31.230.252:34077)
0807 15:17:28.314051 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.314056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.314070 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:17:28.314184 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.314189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.314242 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.314336 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.314849 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.315629 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.316749 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.317145 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.317261 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.320886 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.320889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.320891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.320916 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.322594 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.323093 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.323096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.323098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.323175 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.323476 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.323479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.323481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.323677 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.323774 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.324105 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.324279 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:17:28.328289 19127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8: Bootstrap starting.
I20260807 15:17:28.330947 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.319309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47942)
0807 15:17:28.319559 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.319574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.319646 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.320334 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.320350 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.320619 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.320763 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.320770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.322397 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.322400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.323035 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.323045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.324913 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.324916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.325010 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.325510 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.325563 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.326915 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.329192 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.329195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.329197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.329441 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.329799 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.329803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.329806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.329911 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.330380 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.330384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.330487 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.330580 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.330738 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260807 15:17:28.330950 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.319996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:34077 (local address 127.0.0.1:47948)
0807 15:17:28.320177 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:28.320191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:28.320297 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:28.320703 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:28.320707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:28.320935 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:28.321055 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.321063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.322542 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.322561 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.323372 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:28.323383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.324013 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.324016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:28.324095 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.324864 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:28.324879 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:28.326955 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:28.329370 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.329373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.329375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.329643 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.329976 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:28.329979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:28.329981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:28.330074 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:28.330487 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:28.330490 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:28.330589 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 15:17:28.330671 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:17:28.330799 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 15:17:28.331640 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.319574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.31.230.252:34077)
0807 15:17:28.320002 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.320013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.320028 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:17:28.320090 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.320093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.320154 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.320870 (+ 716us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.320959 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.322281 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.323442 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.324797 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.325308 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.327040 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.327043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.327045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.327064 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.329074 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.329585 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.329589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.329592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.329686 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.330020 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.330025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.330027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.330265 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.331253 (+ 988us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.331327 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.331444 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":949,"server-negotiator.queue_time_us":326,"thread_start_us":164,"threads_started":1}
Aug 07 15:17:28 dist-test-slave-gkw7 krb5kdc[19010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:17:28.331709 19127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:28.332624 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.319974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.31.230.252:34077)
0807 15:17:28.320273 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.320282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.320295 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:17:28.320397 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.320399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.320476 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.321172 (+ 696us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.321271 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.322401 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.323501 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.323905 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.324291 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.327109 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.327112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.327114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.327137 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.329247 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.329772 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.329775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.329777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.329864 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.330190 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.330193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.330195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.330377 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.331618 (+ 1241us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.332320 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.332427 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":231,"spinlock_wait_cycles":7424,"thread_start_us":90,"threads_started":1}
I20260807 15:17:28.332669 19127 log.cc:824] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:28.335510 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: Bootstrap starting.
I20260807 15:17:28.335634 19329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: Bootstrap starting.
I20260807 15:17:28.335897 19127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8: No bootstrap required, opened a new log
I20260807 15:17:28.336297 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.323810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.31.230.253:41725)
0807 15:17:28.323920 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.323925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.324015 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:17:28.324493 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.324502 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.324558 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.324629 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.325890 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.326431 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.328453 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.328855 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.329035 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.332199 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.332202 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.332205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.332228 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.333972 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.334836 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.334839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.334841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.334928 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.335272 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.335276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.335277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.335486 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.335650 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.336033 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.336163 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:17:28.338187 19329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:28.338922 19232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:28.339475 19329 log.cc:824] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:28.339866 19232 log.cc:824] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:28.340164 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:17:28.341603 19329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: No bootstrap required, opened a new log
I20260807 15:17:28.342337 19127 raft_consensus.cc:348] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.342625 19127 raft_consensus.cc:374] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:28.342700 19127 raft_consensus.cc:729] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b91460625fe846e5a01fcea51f5130f8, State: Initialized, Role: FOLLOWER
I20260807 15:17:28.343297 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: No bootstrap required, opened a new log
I20260807 15:17:28.343215 19127 consensus_queue.cc:260] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.344743 19127 sys_catalog.cc:564] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:28.344832 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.344946 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.347450 19232 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.347707 19232 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:28.347779 19232 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d3d70b47f04f6796746556a42ac08d, State: Initialized, Role: FOLLOWER
I20260807 15:17:28.348260 19232 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.349766 19232 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:28.350704 19372 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.350858 19372 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.354830 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.343381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47958 (local address 127.31.230.252:34077)
0807 15:17:28.343498 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.343505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.344627 (+ 1122us) server_negotiation.cc:407] Connection header received
0807 15:17:28.345233 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.345236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.345287 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.345336 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.346366 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.347179 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.348808 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.349148 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.349243 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.350898 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.350904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.350909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.350936 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.352608 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.353090 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.353093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.353096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.353183 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.353468 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.353472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.353473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.353727 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.353895 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.354530 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.354657 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:17:28.350018 19029 master_runner.cc:186] Error getting master registration for 127.31.230.253:41725: OK, instance_id { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" instance_seqno: 1786115848058126 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:17:28.356066 19029 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 15:17:28.356880 19371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:17:28.356961 19371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:17:28.359536 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.342898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.31.230.253:41725)
0807 15:17:28.343012 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.343017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.344308 (+ 1291us) server_negotiation.cc:407] Connection header received
0807 15:17:28.344426 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.344429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.344485 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.344560 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.345786 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.346543 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.347833 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.348143 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.348221 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.351515 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.351521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.351527 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.351557 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.353609 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.354561 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.354567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.354569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.354673 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.358199 (+ 3526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.358202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.358204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.358436 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.358545 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.358948 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.359368 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:17:28.363680 19329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.364018 19329 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:28.364117 19329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2085d6b3b0a6466e9617f00d72a80669, State: Initialized, Role: FOLLOWER
I20260807 15:17:28.364753 19329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.362298 19136 master_runner.cc:186] Error getting master registration for 127.31.230.252:34077: OK, instance_id { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" instance_seqno: 1786115848282439 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:17:28.365775 19136 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:17:28.365845 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:28.342391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.31.230.254:34221)
0807 15:17:28.342510 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:28.342515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:28.343992 (+ 1477us) server_negotiation.cc:407] Connection header received
0807 15:17:28.344132 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:28.344135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:28.344188 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:28.344277 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:28.356219 (+ 11942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.356944 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.358189 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:28.358520 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:28.358632 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:28.361506 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:28.361510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:28.361513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:28.361568 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:28.363577 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.364098 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.364101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.364103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.364194 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:28.364642 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:28.364645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:28.364647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:28.364874 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:28.364985 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:28.365555 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 15:17:28.365689 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:17:28.366042 19390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.366201 19390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.366845 19329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:17:28.372334 19387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:17:28.372418 19387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 15:17:28.374394 19405 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:17:28.374506 19405 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:17:28.376310 19240 master_runner.cc:263] No leader master found from master 2085d6b3b0a6466e9617f00d72a80669
I20260807 15:17:28.376386 19240 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:17:28.465754 19353 raft_consensus.cc:482] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:17:28.465993 19353 raft_consensus.cc:504] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.467298 19353 leader_election.cc:288] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6d3d70b47f04f6796746556a42ac08d (127.31.230.253:41725), 2085d6b3b0a6466e9617f00d72a80669 (127.31.230.252:34077)
I20260807 15:17:28.468366 19196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b91460625fe846e5a01fcea51f5130f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d3d70b47f04f6796746556a42ac08d" is_pre_election: true
I20260807 15:17:28.468716 19196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b91460625fe846e5a01fcea51f5130f8 in term 0.
I20260807 15:17:28.469468 19296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b91460625fe846e5a01fcea51f5130f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2085d6b3b0a6466e9617f00d72a80669" is_pre_election: true
I20260807 15:17:28.469807 19296 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b91460625fe846e5a01fcea51f5130f8 in term 0.
I20260807 15:17:28.470361 19057 leader_election.cc:302] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2085d6b3b0a6466e9617f00d72a80669, b91460625fe846e5a01fcea51f5130f8; no voters:
I20260807 15:17:28.470927 19353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:17:28.471024 19353 raft_consensus.cc:482] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:17:28.471074 19353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:28.472432 19353 raft_consensus.cc:504] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.472787 19353 leader_election.cc:288] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [CANDIDATE]: Term 1 election: Requested vote from peers b6d3d70b47f04f6796746556a42ac08d (127.31.230.253:41725), 2085d6b3b0a6466e9617f00d72a80669 (127.31.230.252:34077)
I20260807 15:17:28.473635 19196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b91460625fe846e5a01fcea51f5130f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d3d70b47f04f6796746556a42ac08d"
I20260807 15:17:28.473767 19196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:28.475039 19196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b91460625fe846e5a01fcea51f5130f8 in term 1.
I20260807 15:17:28.473456 19296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b91460625fe846e5a01fcea51f5130f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2085d6b3b0a6466e9617f00d72a80669"
I20260807 15:17:28.475390 19296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:28.476724 19296 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b91460625fe846e5a01fcea51f5130f8 in term 1.
I20260807 15:17:28.477066 19057 leader_election.cc:302] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2085d6b3b0a6466e9617f00d72a80669, b91460625fe846e5a01fcea51f5130f8; no voters:
I20260807 15:17:28.477403 19353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:28.477689 19353 raft_consensus.cc:686] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [term 1 LEADER]: Becoming Leader. State: Replica: b91460625fe846e5a01fcea51f5130f8, State: Running, Role: LEADER
I20260807 15:17:28.478044 19353 consensus_queue.cc:237] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } }
I20260807 15:17:28.479934 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b91460625fe846e5a01fcea51f5130f8. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.480055 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:28.484880 19413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:28.485965 19413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:28.492327 19296 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [term 1 FOLLOWER]: Refusing update from remote peer b91460625fe846e5a01fcea51f5130f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:28.493165 19409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:28.501042 19390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b91460625fe846e5a01fcea51f5130f8. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.501230 19390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.502537 19196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [term 1 FOLLOWER]: Refusing update from remote peer b91460625fe846e5a01fcea51f5130f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:28.503238 19409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:28.508230 19372 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b91460625fe846e5a01fcea51f5130f8. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.508391 19372 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.520002 19409 sys_catalog.cc:455] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.520226 19409 sys_catalog.cc:458] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:28.521864 19372 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.522017 19372 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.522490 19414 mvcc.cc:205] Tried to move back new op lower bound from 7315930515415019520 to 7315930515369570304. Current Snapshot: MvccSnapshot[applied={T|T < 7315930515415019520}]
I20260807 15:17:28.523650 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.523782 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:28.525163 19390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b91460625fe846e5a01fcea51f5130f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b91460625fe846e5a01fcea51f5130f8" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 34221 } } peers { permanent_uuid: "b6d3d70b47f04f6796746556a42ac08d" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 41725 } } peers { permanent_uuid: "2085d6b3b0a6466e9617f00d72a80669" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 34077 } } }
I20260807 15:17:28.525293 19390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:17:28.526068 19417 mvcc.cc:205] Tried to move back new op lower bound from 7315930515415019520 to 7315930515369570304. Current Snapshot: MvccSnapshot[applied={T|T < 7315930515415019520}]
I20260807 15:17:28.526250 19415 mvcc.cc:205] Tried to move back new op lower bound from 7315930515415019520 to 7315930515369570304. Current Snapshot: MvccSnapshot[applied={T|T < 7315930515415019520}]
I20260807 15:17:28.533378 19413 catalog_manager.cc:1347] Generated new cluster ID: d208768a7dae4f9db115cce16c53b076
I20260807 15:17:28.533466 19413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:28.554878 19413 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:17:28.556190 19413 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:28.565371 19413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b91460625fe846e5a01fcea51f5130f8: Generated new TSK 0
I20260807 15:17:28.566622 19413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:17:29.373847 19387 catalog_manager.cc:1259] Loaded cluster ID: d208768a7dae4f9db115cce16c53b076
I20260807 15:17:29.373939 19387 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: loading cluster ID for follower catalog manager: success
I20260807 15:17:29.375749 19405 catalog_manager.cc:1259] Loaded cluster ID: d208768a7dae4f9db115cce16c53b076
I20260807 15:17:29.375829 19405 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: loading cluster ID for follower catalog manager: success
I20260807 15:17:29.376222 19387 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: acquiring CA information for follower catalog manager: success
I20260807 15:17:29.377708 19387 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b6d3d70b47f04f6796746556a42ac08d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:17:29.377943 19405 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: acquiring CA information for follower catalog manager: success
I20260807 15:17:29.378931 19405 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2085d6b3b0a6466e9617f00d72a80669: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:17:29.627990 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19029
I20260807 15:17:29.642395 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19136
I20260807 15:17:29.657122 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19240
2026-08-07T15:17:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3413 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:17:29 dist-test-slave-gkw7 krb5kdc[19440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:17:29 dist-test-slave-gkw7 krb5kdc[19440](info): set up 2 sockets
Aug 07 15:17:29 dist-test-slave-gkw7 krb5kdc[19440](info): commencing operation
krb5kdc: starting...
W20260807 15:17:30.897998 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:17:30 dist-test-slave-gkw7 krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:17:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:17:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:17:31.037417 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39253
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:44337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:39253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:31.154412 19463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:31.154699 19463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:31.154742 19463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:31.158941 19463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:31.159020 19463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:31.159040 19463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:31.159063 19463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:31.159080 19463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:31.164629 19463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:44337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:39253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:31.165891 19463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:31.167018 19463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:31.176039 19471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:31.176039 19470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:31.176067 19473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:31.176331 19463 server_base.cc:1034] running on GCE node
I20260807 15:17:31.176990 19463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:31.178224 19463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:31.179425 19463 hybrid_clock.cc:648] HybridClock initialized: now 1786115851179381 us; error 57 us; skew 500 ppm
Aug 07 15:17:31 dist-test-slave-gkw7 krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115851, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:31.182473 19463 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:31.183398 19463 webserver.cc:460] Webserver started at http://127.31.230.254:39909/ using document root <none> and password file <none>
I20260807 15:17:31.184003 19463 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:31.184095 19463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:31.184327 19463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:31.186504 19463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "b56e421057ce4ac38da1c4dab581d672"
format_stamp: "Formatted at 2026-08-07 15:17:31 on dist-test-slave-gkw7"
server_key: "d4ae3d0d2217f6c24d3cb0ea4c8858f2"
server_key_iv: "a38127a1b264b1054106ae959b79d4e7"
server_key_version: "encryptionkey@0"
I20260807 15:17:31.187079 19463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "b56e421057ce4ac38da1c4dab581d672"
format_stamp: "Formatted at 2026-08-07 15:17:31 on dist-test-slave-gkw7"
server_key: "d4ae3d0d2217f6c24d3cb0ea4c8858f2"
server_key_iv: "a38127a1b264b1054106ae959b79d4e7"
server_key_version: "encryptionkey@0"
I20260807 15:17:31.191125 19463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:17:31.193687 19483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:31.194993 19463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:17:31.195145 19463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "b56e421057ce4ac38da1c4dab581d672"
format_stamp: "Formatted at 2026-08-07 15:17:31 on dist-test-slave-gkw7"
server_key: "d4ae3d0d2217f6c24d3cb0ea4c8858f2"
server_key_iv: "a38127a1b264b1054106ae959b79d4e7"
server_key_version: "encryptionkey@0"
I20260807 15:17:31.195263 19463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:31.211305 19463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:31.214635 19463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:31.214854 19463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:31.222486 19463 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:39253
I20260807 15:17:31.222512 19552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:39253 every 8 connection(s)
I20260807 15:17:31.223448 19463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:31.225639 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19463
I20260807 15:17:31.225797 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:17:31.226099 32667 external_mini_cluster.cc:1428] Setting key fe841727083ddce867169ac066a272d8
I20260807 15:17:31.226449 19554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:31.233942 19554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672: Bootstrap starting.
Aug 07 15:17:31 dist-test-slave-gkw7 krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:31.236756 19554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:31.238058 19554 log.cc:824] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:31.240254 19554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672: No bootstrap required, opened a new log
I20260807 15:17:31.241777 19558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:31.227597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54424 (local address 127.31.230.254:39253)
0807 15:17:31.228066 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:31.228085 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:31.228129 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:17:31.228890 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:31.228914 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:31.229364 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:31.229779 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:31.230504 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:31.231986 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:31.232900 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:31.233656 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:31.233823 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:31.236551 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:31.236561 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:31.236582 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:31.236619 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:31.238771 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:31.239275 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:31.239281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:31.239285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:31.239400 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:31.239710 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:31.239714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:31.239716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:31.240125 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:31.240342 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:31.240647 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 15:17:31.240959 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":128,"threads_started":1}
I20260807 15:17:31.244959 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:17:31.245147 19554 raft_consensus.cc:348] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56e421057ce4ac38da1c4dab581d672" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39253 } }
I20260807 15:17:31.245415 19554 raft_consensus.cc:374] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:31.245514 19554 raft_consensus.cc:729] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56e421057ce4ac38da1c4dab581d672, State: Initialized, Role: FOLLOWER
I20260807 15:17:31.246093 19554 consensus_queue.cc:260] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56e421057ce4ac38da1c4dab581d672" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39253 } }
I20260807 15:17:31.246294 19554 raft_consensus.cc:388] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:31.246369 19554 raft_consensus.cc:482] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:31.246479 19554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:31.247540 19554 raft_consensus.cc:504] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56e421057ce4ac38da1c4dab581d672" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39253 } }
I20260807 15:17:31.247929 19554 leader_election.cc:302] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b56e421057ce4ac38da1c4dab581d672; no voters:
I20260807 15:17:31.248301 19554 leader_election.cc:288] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:17:31.248464 19560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:31.248706 19560 raft_consensus.cc:686] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [term 1 LEADER]: Becoming Leader. State: Replica: b56e421057ce4ac38da1c4dab581d672, State: Running, Role: LEADER
I20260807 15:17:31.249311 19560 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56e421057ce4ac38da1c4dab581d672" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39253 } }
I20260807 15:17:31.249737 19554 sys_catalog.cc:564] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:31.251305 19564 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56e421057ce4ac38da1c4dab581d672. Latest consensus state: current_term: 1 leader_uuid: "b56e421057ce4ac38da1c4dab581d672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56e421057ce4ac38da1c4dab581d672" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39253 } } }
I20260807 15:17:31.251320 19563 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b56e421057ce4ac38da1c4dab581d672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56e421057ce4ac38da1c4dab581d672" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39253 } } }
I20260807 15:17:31.251464 19563 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:31.251466 19564 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:31.252470 19579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:31.255265 19579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:31.260612 19579 catalog_manager.cc:1347] Generated new cluster ID: 2ced0200cea941a0b151238f2dc7c046
I20260807 15:17:31.260695 19579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:31.285808 19579 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:17:31.287386 19579 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:31.296803 19579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b56e421057ce4ac38da1c4dab581d672: Generated new TSK 0
I20260807 15:17:31.297848 19579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:17:31.352451 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19463
2026-08-07T15:17:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1697 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:17:31 dist-test-slave-gkw7 krb5kdc[19594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:17:31 dist-test-slave-gkw7 krb5kdc[19594](info): set up 2 sockets
Aug 07 15:17:31 dist-test-slave-gkw7 krb5kdc[19594](info): commencing operation
krb5kdc: starting...
W20260807 15:17:32.596906 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:17:32 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:17:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:17:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:17:32.732322 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45363
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:32.846771 19618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:32.847035 19618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:32.847074 19618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:32.851163 19618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:32.851228 19618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:32.851248 19618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:32.851263 19618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:32.851279 19618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:32.856601 19618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:45363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:32.857848 19618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:32.858952 19618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:32.868062 19624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:32.868041 19618 server_base.cc:1034] running on GCE node
W20260807 15:17:32.868041 19623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:32.868041 19626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:32.868882 19618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:32.870198 19618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:32.871387 19618 hybrid_clock.cc:648] HybridClock initialized: now 1786115852871350 us; error 45 us; skew 500 ppm
Aug 07 15:17:32 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:32.874317 19618 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:32.875205 19618 webserver.cc:460] Webserver started at http://127.31.230.254:37427/ using document root <none> and password file <none>
I20260807 15:17:32.875743 19618 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:32.875805 19618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:32.875979 19618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:32.878156 19618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "4e317818ae3d45b78001389f1f44a695"
format_stamp: "Formatted at 2026-08-07 15:17:32 on dist-test-slave-gkw7"
server_key: "6008723d28c89b9808ccc6dfefdd7c0b"
server_key_iv: "9667e5efd165169ceb94b637781aa291"
server_key_version: "encryptionkey@0"
I20260807 15:17:32.878708 19618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "4e317818ae3d45b78001389f1f44a695"
format_stamp: "Formatted at 2026-08-07 15:17:32 on dist-test-slave-gkw7"
server_key: "6008723d28c89b9808ccc6dfefdd7c0b"
server_key_iv: "9667e5efd165169ceb94b637781aa291"
server_key_version: "encryptionkey@0"
I20260807 15:17:32.882799 19618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:17:32.885857 19637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:32.887329 19618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:17:32.887516 19618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "4e317818ae3d45b78001389f1f44a695"
format_stamp: "Formatted at 2026-08-07 15:17:32 on dist-test-slave-gkw7"
server_key: "6008723d28c89b9808ccc6dfefdd7c0b"
server_key_iv: "9667e5efd165169ceb94b637781aa291"
server_key_version: "encryptionkey@0"
I20260807 15:17:32.887663 19618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:32.907375 19618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:32.911269 19618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:32.911531 19618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:32.920053 19618 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:45363
I20260807 15:17:32.920361 19708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:45363 every 8 connection(s)
I20260807 15:17:32.921659 19618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:32.924804 19709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:32.930202 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19618
I20260807 15:17:32.930354 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:17:32.930663 32667 external_mini_cluster.cc:1428] Setting key 4a22581702e2b1b222e6ecf5c5f75621
I20260807 15:17:32.930950 19709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap starting.
I20260807 15:17:32.933851 19709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:32.934779 19709 log.cc:824] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:32.937162 19709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: No bootstrap required, opened a new log
Aug 07 15:17:32 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:32.941648 19709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:32.942022 19709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:32.942112 19709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e317818ae3d45b78001389f1f44a695, State: Initialized, Role: FOLLOWER
I20260807 15:17:32.942705 19709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:32.942904 19709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:32.942991 19709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:32.943122 19709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:32.944399 19709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:32.944801 19709 leader_election.cc:302] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e317818ae3d45b78001389f1f44a695; no voters:
I20260807 15:17:32.945231 19709 leader_election.cc:288] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:17:32.945385 19716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:32.945683 19716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 LEADER]: Becoming Leader. State: Replica: 4e317818ae3d45b78001389f1f44a695, State: Running, Role: LEADER
I20260807 15:17:32.946041 19716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:32.946728 19709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:32.947335 19716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e317818ae3d45b78001389f1f44a695. Latest consensus state: current_term: 1 leader_uuid: "4e317818ae3d45b78001389f1f44a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } } }
I20260807 15:17:32.947530 19716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:32.947796 19717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e317818ae3d45b78001389f1f44a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } } }
I20260807 15:17:32.947948 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:32.932172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.31.230.254:45363)
0807 15:17:32.932650 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:32.932670 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:32.932714 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:17:32.933567 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:32.933603 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:32.934050 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:32.934442 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:32.935193 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:32.936645 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:32.937713 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:32.938406 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:32.938576 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:32.941517 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:32.941524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:32.941563 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:32.941606 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:32.943926 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:32.944428 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:32.944435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:32.944441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:32.944607 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:32.944907 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:32.944913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:32.944915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:32.945490 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:32.946357 (+ 867us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:32.946516 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:17:32.946858 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":138,"threads_started":1}
I20260807 15:17:32.947959 19717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:32.948058 19731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:32.951994 19731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:32.958640 19731 catalog_manager.cc:1347] Generated new cluster ID: 4439192a7ac24836881b1844e4fa5ae8
I20260807 15:17:32.958737 19731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:32.980170 19731 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:17:32.981709 19731 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:32.998574 19731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Generated new TSK 0
I20260807 15:17:32.999367 19731 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:17:33.038913 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45363
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:33.158313 19747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:33.158596 19747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:33.158640 19747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:33.162719 19747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:33.162788 19747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:33.162849 19747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:17:33.168583 19747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:33.169688 19747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:33.170710 19747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:33.180826 19758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:33.180830 19764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:33.180989 19747 server_base.cc:1034] running on GCE node
W20260807 15:17:33.181165 19759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:33.181643 19747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:33.182307 19747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:33.183499 19747 hybrid_clock.cc:648] HybridClock initialized: now 1786115853183479 us; error 38 us; skew 500 ppm
Aug 07 15:17:33 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:33.186326 19747 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:17:33.187191 19747 webserver.cc:460] Webserver started at http://127.31.230.193:33613/ using document root <none> and password file <none>
I20260807 15:17:33.187779 19747 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:33.187873 19747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:33.188112 19747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:33.190217 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "62f23dffbe5a438691c5bb9ba6d343b8"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "21d3880d80810ebd2b423b91c9c00636"
server_key_iv: "0723077cc08c62bbff2544709a8f6130"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.190783 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "62f23dffbe5a438691c5bb9ba6d343b8"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "21d3880d80810ebd2b423b91c9c00636"
server_key_iv: "0723077cc08c62bbff2544709a8f6130"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.194684 19747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:17:33.197258 19774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.198613 19747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:17:33.198755 19747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "62f23dffbe5a438691c5bb9ba6d343b8"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "21d3880d80810ebd2b423b91c9c00636"
server_key_iv: "0723077cc08c62bbff2544709a8f6130"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.198869 19747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:33.210427 19747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:33.213508 19747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:33.213809 19747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:33.214417 19747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:33.215256 19747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:33.215327 19747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.215397 19747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:33.215452 19747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.224900 19747 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:36501
I20260807 15:17:33.224921 19938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:36501 every 8 connection(s)
I20260807 15:17:33.225996 19747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:17:33.226733 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19747
I20260807 15:17:33.226883 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:17:33.227250 32667 external_mini_cluster.cc:1428] Setting key 0bf9a227aaab2497016811bbe3ea2c1c
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Aug 07 15:17:33 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:17:33.244863 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.228127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:54769 (local address 127.31.230.254:45363)
0807 15:17:33.228252 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.228256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.231314 (+ 3058us) server_negotiation.cc:407] Connection header received
0807 15:17:33.232607 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.232610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.232657 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.232776 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:33.234539 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.235083 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.236975 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.237213 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.237312 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.240301 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:33.240305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:33.240307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:33.240334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:33.242209 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.242972 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.242979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.242981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.243092 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.243480 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.243486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.243491 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.243780 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:33.243893 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.244425 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.244603 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:17:33.245522 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.228555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:54769)
0807 15:17:33.231145 (+ 2590us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.231193 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.232390 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.232966 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.232977 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.233610 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:33.234351 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.234382 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.235224 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.235228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.236833 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.236845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.237397 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.237402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.237659 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.238311 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:33.238332 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:33.240097 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:33.242393 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:33.242400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:33.242418 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:33.242823 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:33.243201 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:33.243203 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:33.243204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:33.243366 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:33.243889 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:33.243894 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:33.244147 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.244321 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.244598 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2378,"thread_start_us":156,"threads_started":1}
I20260807 15:17:33.246920 19940 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45363
I20260807 15:17:33.247237 19940 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:33.247781 19940 heartbeater.cc:511] Master 127.31.230.254:45363 requested a full tablet report, sending...
I20260807 15:17:33.249504 19660 ts_manager.cc:194] Registered new tserver with Master: 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501)
I20260807 15:17:33.250717 19660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:54769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:17:33.260564 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45363
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:33.379400 19946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:33.379760 19946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:33.379856 19946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:33.383960 19946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:33.384060 19946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:33.384160 19946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:17:33.389739 19946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:33.390964 19946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:33.392119 19946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:33.401882 19955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:33.401944 19959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:33.401909 19946 server_base.cc:1034] running on GCE node
W20260807 15:17:33.402094 19956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:33.402540 19946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:33.403266 19946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:33.404431 19946 hybrid_clock.cc:648] HybridClock initialized: now 1786115853404413 us; error 29 us; skew 500 ppm
Aug 07 15:17:33 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:33.407229 19946 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:17:33.408111 19946 webserver.cc:460] Webserver started at http://127.31.230.194:39751/ using document root <none> and password file <none>
I20260807 15:17:33.408666 19946 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:33.408728 19946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:33.408900 19946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:33.410966 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "3b6d630c16ae4d6c9021875eff089c8a"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "f10f770a59ca97e166c766c19630fdee"
server_key_iv: "be4ee9a412b4c695f45e955cb98d34b6"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.411491 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "3b6d630c16ae4d6c9021875eff089c8a"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "f10f770a59ca97e166c766c19630fdee"
server_key_iv: "be4ee9a412b4c695f45e955cb98d34b6"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.415690 19946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:17:33.418108 19969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.419379 19946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:17:33.419482 19946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3b6d630c16ae4d6c9021875eff089c8a"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "f10f770a59ca97e166c766c19630fdee"
server_key_iv: "be4ee9a412b4c695f45e955cb98d34b6"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.419569 19946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:33.438782 19946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:33.442216 19946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:33.442400 19946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:33.442973 19946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:33.443801 19946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:33.443851 19946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.443892 19946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:33.443907 19946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.453773 19946 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:35395
I20260807 15:17:33.453796 20134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:35395 every 8 connection(s)
I20260807 15:17:33.454743 19946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:17:33.458498 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 19946
I20260807 15:17:33.458660 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:17:33.458978 32667 external_mini_cluster.cc:1428] Setting key db255d2073e0bdcb4ced4cebbc1ad7c4
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Aug 07 15:17:33 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:17:33.475992 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.456767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:52739 (local address 127.31.230.254:45363)
0807 15:17:33.456939 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.456944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.457912 (+ 968us) server_negotiation.cc:407] Connection header received
0807 15:17:33.459694 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.459697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.459778 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.459870 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:33.461453 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.462065 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.463935 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.464352 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.464475 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.471590 (+ 7115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:33.471593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:33.471595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:33.471626 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:33.473397 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.474078 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.474085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.474090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.474189 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.474650 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.474652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.474653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.474851 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:33.474928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.475648 (+ 720us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.475812 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:17:33.477149 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.457178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:52739)
0807 15:17:33.457742 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.457801 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.459444 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.460041 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.460048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.460560 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:33.461291 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.461315 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.462200 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.462207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.463740 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.463753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.464569 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.464575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.464831 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.469722 (+ 4891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:33.469740 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:33.471380 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:33.473519 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:33.473524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:33.473586 (+ 62us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:33.473968 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:33.474309 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:33.474312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:33.474315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:33.474536 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:33.474965 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:33.474975 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:33.475359 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.475598 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.476055 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":120,"threads_started":1}
I20260807 15:17:33.478605 20135 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45363
I20260807 15:17:33.478910 20135 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:33.479542 20135 heartbeater.cc:511] Master 127.31.230.254:45363 requested a full tablet report, sending...
I20260807 15:17:33.480832 19660 ts_manager.cc:194] Registered new tserver with Master: 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395)
I20260807 15:17:33.481756 19660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:52739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:17:33.493077 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45363
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:33.613432 20142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:33.613724 20142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:33.613803 20142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:33.617937 20142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:33.618010 20142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:33.618121 20142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:17:33.623725 20142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:33.625013 20142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:33.626255 20142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:33.637251 20151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:33.637231 20154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:33.637279 20150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:33.637662 20142 server_base.cc:1034] running on GCE node
I20260807 15:17:33.638068 20142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:33.638773 20142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:33.639991 20142 hybrid_clock.cc:648] HybridClock initialized: now 1786115853639962 us; error 49 us; skew 500 ppm
Aug 07 15:17:33 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:33.643093 20142 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:17:33.644065 20142 webserver.cc:460] Webserver started at http://127.31.230.195:34895/ using document root <none> and password file <none>
I20260807 15:17:33.644649 20142 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:33.644713 20142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:33.644896 20142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:33.647176 20142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "0bf29b062a904d6180416e3e6debd964"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "3013ef6839ce30d8ad8966bfa0c88cdb"
server_key_iv: "fe13c5c47282ee7a7ba9b6f730b6a5b7"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.647778 20142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "0bf29b062a904d6180416e3e6debd964"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "3013ef6839ce30d8ad8966bfa0c88cdb"
server_key_iv: "fe13c5c47282ee7a7ba9b6f730b6a5b7"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.652033 20142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:17:33.654671 20165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.655944 20142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:17:33.656054 20142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0bf29b062a904d6180416e3e6debd964"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "3013ef6839ce30d8ad8966bfa0c88cdb"
server_key_iv: "fe13c5c47282ee7a7ba9b6f730b6a5b7"
server_key_version: "encryptionkey@0"
I20260807 15:17:33.656140 20142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:33.683022 20142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:33.686240 20142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:33.686412 20142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:33.687003 20142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:33.687865 20142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:33.687913 20142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.687954 20142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:33.687978 20142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:33.698217 20142 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:41329
I20260807 15:17:33.698282 20325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:41329 every 8 connection(s)
I20260807 15:17:33.699198 20142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:17:33.702173 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 20142
I20260807 15:17:33.702315 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:17:33.702625 32667 external_mini_cluster.cc:1428] Setting key 1a39c54213e41af287a34c958ae2a6f1
Aug 07 15:17:33 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:33.713624 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.701080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:49557 (local address 127.31.230.254:45363)
0807 15:17:33.701268 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.701273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.702197 (+ 924us) server_negotiation.cc:407] Connection header received
0807 15:17:33.703997 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.704000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.704056 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.704175 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:33.705499 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.706142 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.707319 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.707552 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.707669 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.710189 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:33.710191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:33.710194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:33.710218 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:33.711473 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.711999 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.712003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.712004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.712089 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.712445 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.712448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.712450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.712639 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:33.712749 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.713250 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.713385 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:17:33.714443 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.701493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:49557)
0807 15:17:33.702034 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.702079 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.703789 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.704358 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.704365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.704733 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:33.705340 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.705354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.706276 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.706280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.707188 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.707197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.707715 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.707719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.707899 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.708462 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:33.708479 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:33.709986 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:33.711596 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:33.711602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:33.711618 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:33.711884 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:33.712186 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:33.712189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:33.712191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:33.712335 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:33.712747 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:33.712754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:33.712997 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.713193 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.713551 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":157,"threads_started":1}
I20260807 15:17:33.715648 20327 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45363
I20260807 15:17:33.715929 20327 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:33.716460 20327 heartbeater.cc:511] Master 127.31.230.254:45363 requested a full tablet report, sending...
I20260807 15:17:33.717582 19660 ts_manager.cc:194] Registered new tserver with Master: 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:33.718174 19660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:49557
I20260807 15:17:33.729054 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:17:33.739210 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.731368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.31.230.254:45363)
0807 15:17:33.731545 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.731551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.731717 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:17:33.731879 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.731888 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.731934 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.732020 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:33.732778 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.733360 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.734318 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.734540 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.734658 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.736139 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:33.736142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:33.736144 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:33.736168 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:33.737368 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.737853 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.737856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.737858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.737943 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:33.738226 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:33.738229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:33.738231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:33.738475 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:33.738624 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.738902 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.739034 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:17:33.742244 19660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:17:33.745282 19660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:17:33.760932 20348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.754981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.31.230.194:35395)
0807 15:17:33.755548 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.755565 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.755593 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:17:33.755693 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.755697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.755944 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.756101 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:33.756689 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.757702 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.759363 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.760214 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.760355 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.760496 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.760669 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.760775 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":164,"threads_started":1}
I20260807 15:17:33.760914 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.754852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35395 (local address 127.0.0.1:40140)
0807 15:17:33.755328 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.755378 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.755518 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.756336 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.756341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.756368 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:33.756522 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.756533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.757855 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.757858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.759222 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.759233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.760343 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.760346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.760454 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.760492 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.760633 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.760764 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":90,"threads_started":1}
I20260807 15:17:33.761689 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.755947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41329 (local address 127.0.0.1:46832)
0807 15:17:33.756288 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.756301 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.756386 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.757158 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.757161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.757178 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:33.757313 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.757320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.758808 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.758815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.759856 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.759868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.761203 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.761206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.761308 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.761332 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.761415 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.761497 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":22,"thread_start_us":87,"threads_started":1}
I20260807 15:17:33.761751 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.756384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.31.230.195:41329)
0807 15:17:33.756756 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.756767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.756787 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:17:33.756853 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.756857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.757009 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.757467 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:33.757651 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.758682 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.759980 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.760883 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.761003 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.761140 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.761469 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.761605 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":145,"threads_started":1}
I20260807 15:17:33.762368 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.755595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36501 (local address 127.0.0.1:56884)
0807 15:17:33.755867 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.755881 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.755972 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.757069 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.757072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.757092 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:33.757188 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.757193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.758594 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.758598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.759726 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.759772 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.761991 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.761994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.762060 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.762076 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.762177 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.762247 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":25,"thread_start_us":117,"threads_started":1}
I20260807 15:17:33.762445 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.755900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.31.230.193:36501)
0807 15:17:33.756614 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.756623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.756646 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:17:33.756723 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.756728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.756923 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.757076 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:33.757314 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.758470 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.759948 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.760779 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.761785 (+ 1006us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.761922 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.762212 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.762309 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":79,"threads_started":1}
I20260807 15:17:33.764681 20043 tablet_service.cc:1467] Processing CreateTablet for tablet fe5cbe58d5404a0881aeaa471866a68d (DEFAULT_TABLE table=test-table [id=b6b6b64725b5493bbe2c2d8d2fafdc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:33.765142 19846 tablet_service.cc:1467] Processing CreateTablet for tablet fe5cbe58d5404a0881aeaa471866a68d (DEFAULT_TABLE table=test-table [id=b6b6b64725b5493bbe2c2d8d2fafdc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:33.765304 20230 tablet_service.cc:1467] Processing CreateTablet for tablet fe5cbe58d5404a0881aeaa471866a68d (DEFAULT_TABLE table=test-table [id=b6b6b64725b5493bbe2c2d8d2fafdc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:33.765930 20043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5cbe58d5404a0881aeaa471866a68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:33.766273 19846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5cbe58d5404a0881aeaa471866a68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:33.766439 20230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5cbe58d5404a0881aeaa471866a68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:33.772415 20354 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Bootstrap starting.
I20260807 15:17:33.774020 20357 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap starting.
I20260807 15:17:33.774353 20355 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Bootstrap starting.
I20260807 15:17:33.776268 20357 tablet_bootstrap.cc:654] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:33.776496 20355 tablet_bootstrap.cc:654] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:33.776517 20354 tablet_bootstrap.cc:654] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:33.777232 20355 log.cc:824] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:33.777242 20354 log.cc:824] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:33.777242 20357 log.cc:824] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:33.779141 20355 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: No bootstrap required, opened a new log
I20260807 15:17:33.779289 20357 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: No bootstrap required, opened a new log
I20260807 15:17:33.779289 20354 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: No bootstrap required, opened a new log
I20260807 15:17:33.779420 20355 ts_tablet_manager.cc:1397] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:17:33.779479 20357 ts_tablet_manager.cc:1397] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:17:33.779479 20354 ts_tablet_manager.cc:1397] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260807 15:17:33.782855 20355 raft_consensus.cc:348] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.783190 20355 raft_consensus.cc:374] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:33.783268 20355 raft_consensus.cc:729] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6d630c16ae4d6c9021875eff089c8a, State: Initialized, Role: FOLLOWER
I20260807 15:17:33.783582 20354 raft_consensus.cc:348] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.783740 20355 consensus_queue.cc:260] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.783849 20354 raft_consensus.cc:374] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:33.783910 20354 raft_consensus.cc:729] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf29b062a904d6180416e3e6debd964, State: Initialized, Role: FOLLOWER
I20260807 15:17:33.783914 20357 raft_consensus.cc:348] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.784169 20357 raft_consensus.cc:374] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:33.784232 20357 raft_consensus.cc:729] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f23dffbe5a438691c5bb9ba6d343b8, State: Initialized, Role: FOLLOWER
I20260807 15:17:33.784413 20354 consensus_queue.cc:260] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.784646 20135 heartbeater.cc:503] Master 127.31.230.254:45363 was elected leader, sending a full tablet report...
I20260807 15:17:33.784616 20357 consensus_queue.cc:260] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.784874 20355 ts_tablet_manager.cc:1428] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:17:33.785415 20354 ts_tablet_manager.cc:1428] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260807 15:17:33.785449 20357 ts_tablet_manager.cc:1428] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:17:33.785773 20327 heartbeater.cc:503] Master 127.31.230.254:45363 was elected leader, sending a full tablet report...
I20260807 15:17:33.785773 19940 heartbeater.cc:503] Master 127.31.230.254:45363 was elected leader, sending a full tablet report...
I20260807 15:17:33.873340 20362 raft_consensus.cc:482] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:17:33.873620 20362 raft_consensus.cc:504] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.875145 20362 leader_election.cc:288] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501), 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:33.879024 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.875358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36501 (local address 127.31.230.194:37667)
0807 15:17:33.875572 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.875591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.875706 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.876045 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.876048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.876070 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:33.876195 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.876201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.876999 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.877003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.877888 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.877898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.878573 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.878577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.878684 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.878714 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.878811 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.878889 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 15:17:33.879024 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.875484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:37667 (local address 127.31.230.193:36501)
0807 15:17:33.875638 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.875645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.875659 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:17:33.875859 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.875866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.875928 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.876014 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:33.876343 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.876881 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.878020 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.878452 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.878583 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.878638 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.878829 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.878900 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:17:33.879393 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.875693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41329 (local address 127.31.230.194:53719)
0807 15:17:33.876102 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.876114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:33.876242 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:33.876531 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:33.876534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:33.876550 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:33.876677 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.876682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.877413 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.877417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.878300 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:33.878309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.878964 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.878968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:33.879067 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.879095 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:17:33.879185 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:17:33.879257 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":161,"threads_started":1}
I20260807 15:17:33.879655 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.875843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:53719 (local address 127.31.230.195:41329)
0807 15:17:33.875982 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.875987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.876202 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:17:33.876377 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.876380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.876431 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.876516 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:33.876785 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.877291 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.878427 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.878835 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.878964 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.879012 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.879333 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.879417 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:17:33.879696 19871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5cbe58d5404a0881aeaa471866a68d" candidate_uuid: "3b6d630c16ae4d6c9021875eff089c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" is_pre_election: true
I20260807 15:17:33.880004 19871 raft_consensus.cc:2454] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b6d630c16ae4d6c9021875eff089c8a in term 0.
I20260807 15:17:33.880246 20254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5cbe58d5404a0881aeaa471866a68d" candidate_uuid: "3b6d630c16ae4d6c9021875eff089c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf29b062a904d6180416e3e6debd964" is_pre_election: true
I20260807 15:17:33.880543 20254 raft_consensus.cc:2454] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b6d630c16ae4d6c9021875eff089c8a in term 0.
I20260807 15:17:33.880517 19973 leader_election.cc:302] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b6d630c16ae4d6c9021875eff089c8a, 62f23dffbe5a438691c5bb9ba6d343b8; no voters:
I20260807 15:17:33.880846 20362 raft_consensus.cc:2781] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:17:33.880959 20362 raft_consensus.cc:482] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:17:33.881038 20362 raft_consensus.cc:3037] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:33.882190 20362 raft_consensus.cc:504] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.882536 20362 leader_election.cc:288] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [CANDIDATE]: Term 1 election: Requested vote from peers 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501), 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:33.883004 19871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5cbe58d5404a0881aeaa471866a68d" candidate_uuid: "3b6d630c16ae4d6c9021875eff089c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62f23dffbe5a438691c5bb9ba6d343b8"
I20260807 15:17:33.883004 20254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5cbe58d5404a0881aeaa471866a68d" candidate_uuid: "3b6d630c16ae4d6c9021875eff089c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf29b062a904d6180416e3e6debd964"
I20260807 15:17:33.883133 20254 raft_consensus.cc:3037] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:33.883143 19871 raft_consensus.cc:3037] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:33.884462 19871 raft_consensus.cc:2454] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b6d630c16ae4d6c9021875eff089c8a in term 1.
I20260807 15:17:33.884462 20254 raft_consensus.cc:2454] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b6d630c16ae4d6c9021875eff089c8a in term 1.
I20260807 15:17:33.884747 19973 leader_election.cc:302] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf29b062a904d6180416e3e6debd964, 3b6d630c16ae4d6c9021875eff089c8a; no voters:
I20260807 15:17:33.884910 20362 raft_consensus.cc:2781] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:33.885144 20362 raft_consensus.cc:686] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 1 LEADER]: Becoming Leader. State: Replica: 3b6d630c16ae4d6c9021875eff089c8a, State: Running, Role: LEADER
I20260807 15:17:33.885475 20362 consensus_queue.cc:237] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:33.888988 19660 catalog_manager.cc:5495] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194). New cstate: current_term: 1 leader_uuid: "3b6d630c16ae4d6c9021875eff089c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } health_report { overall_health: UNKNOWN } } }
I20260807 15:17:33.912508 20348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.908935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.31.230.194:35395)
0807 15:17:33.909087 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:33.909092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:33.909106 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:17:33.909306 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:33.909308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:33.909367 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:33.909485 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:17:33.909828 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.910358 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.911192 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:33.911411 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:33.911488 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:33.911757 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:17:33.912165 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:17:33.912278 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:33.912327 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:17:33.912378 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:17:33.918383 20254 raft_consensus.cc:1261] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Refusing update from remote peer 3b6d630c16ae4d6c9021875eff089c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:33.918412 19871 raft_consensus.cc:1261] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Refusing update from remote peer 3b6d630c16ae4d6c9021875eff089c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:33.919137 20362 consensus_queue.cc:1035] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:33.919468 20368 consensus_queue.cc:1035] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:33.927553 20373 mvcc.cc:205] Tried to move back new op lower bound from 7315930537642315776 to 7315930537518108672. Current Snapshot: MvccSnapshot[applied={T|T < 7315930537642315776}]
I20260807 15:17:33.938217 19659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:33.938412 19659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:33.940941 19659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Sending DeleteTablet for 3 replicas of tablet fe5cbe58d5404a0881aeaa471866a68d
I20260807 15:17:33.941807 19846 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5cbe58d5404a0881aeaa471866a68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:33 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:56884
I20260807 15:17:33.941882 20043 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5cbe58d5404a0881aeaa471866a68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:33 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:40140
I20260807 15:17:33.941892 20230 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5cbe58d5404a0881aeaa471866a68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:33 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:46832
I20260807 15:17:33.942373 20384 tablet_replica.cc:330] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: stopping tablet replica
I20260807 15:17:33.942549 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19747
I20260807 15:17:33.944856 20384 raft_consensus.cc:2229] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:17:33.945313 20384 raft_consensus.cc:2258] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:33.949800 20384 ts_tablet_manager.cc:1905] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:33.954523 20384 ts_tablet_manager.cc:1918] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:33.954624 20384 log.cc:1196] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/fe5cbe58d5404a0881aeaa471866a68d
I20260807 15:17:33.954905 20384 ts_tablet_manager.cc:1939] T fe5cbe58d5404a0881aeaa471866a68d P 3b6d630c16ae4d6c9021875eff089c8a: Deleting consensus metadata
I20260807 15:17:33.956192 20385 tablet_replica.cc:330] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: stopping tablet replica
I20260807 15:17:33.956189 19642 catalog_manager.cc:4841] TS 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395): tablet fe5cbe58d5404a0881aeaa471866a68d (table test-table [id=b6b6b64725b5493bbe2c2d8d2fafdc1c]) successfully deleted
I20260807 15:17:33.956542 20385 raft_consensus.cc:2229] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:17:33.956856 20385 raft_consensus.cc:2258] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:33.958287 20385 ts_tablet_manager.cc:1905] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:33.960739 20385 ts_tablet_manager.cc:1918] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:33.960853 20385 log.cc:1196] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/fe5cbe58d5404a0881aeaa471866a68d
I20260807 15:17:33.961158 20385 ts_tablet_manager.cc:1939] T fe5cbe58d5404a0881aeaa471866a68d P 0bf29b062a904d6180416e3e6debd964: Deleting consensus metadata
I20260807 15:17:33.962399 19642 catalog_manager.cc:4841] TS 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329): tablet fe5cbe58d5404a0881aeaa471866a68d (table test-table [id=b6b6b64725b5493bbe2c2d8d2fafdc1c]) successfully deleted
W20260807 15:17:33.963933 19642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:36501 (error 108)
I20260807 15:17:33.964875 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19946
I20260807 15:17:33.965010 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:33.964696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36501 (local address 127.0.0.1:56892)
0807 15:17:33.964846 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:33.964917 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 15:17:33.976575 19642 catalog_manager.cc:4584] TS 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe5cbe58d5404a0881aeaa471866a68d: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36501: connect: Connection refused (error 111)
I20260807 15:17:33.982622 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 20142
I20260807 15:17:33.992173 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 19618
I20260807 15:17:34.004624 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45363
--webserver_interface=127.31.230.254
--webserver_port=37427
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:34.122748 20388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:34.123114 20388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:34.123208 20388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:34.127449 20388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:34.127563 20388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:34.127609 20388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:34.127671 20388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:34.127732 20388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:34.133584 20388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:45363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=37427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:34.134783 20388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:34.135902 20388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:34.151233 20401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.151211 20403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.151211 20400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:34.151496 20388 server_base.cc:1034] running on GCE node
I20260807 15:17:34.152187 20388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:34.153483 20388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:34.154724 20388 hybrid_clock.cc:648] HybridClock initialized: now 1786115854154709 us; error 27 us; skew 500 ppm
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:34.157863 20388 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:34.158910 20388 webserver.cc:460] Webserver started at http://127.31.230.254:37427/ using document root <none> and password file <none>
I20260807 15:17:34.159543 20388 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:34.159672 20388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:34.163810 20388 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 15:17:34.166167 20411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.167440 20388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:17:34.167591 20388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "4e317818ae3d45b78001389f1f44a695"
format_stamp: "Formatted at 2026-08-07 15:17:32 on dist-test-slave-gkw7"
server_key: "6008723d28c89b9808ccc6dfefdd7c0b"
server_key_iv: "9667e5efd165169ceb94b637781aa291"
server_key_version: "encryptionkey@0"
I20260807 15:17:34.168221 20388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:34.187876 20388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:34.191205 20388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:34.191443 20388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:34.199357 20388 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:45363
I20260807 15:17:34.199739 20501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:45363 every 8 connection(s)
I20260807 15:17:34.200601 20388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:34.202090 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 20388
I20260807 15:17:34.202630 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:36501
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:45363
--webserver_port=33613
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:34.206357 20502 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:17:34.208166 20502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap starting.
I20260807 15:17:34.225826 20502 log.cc:824] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:34.235118 20502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:34.235520 20502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap complete.
I20260807 15:17:34.239907 20502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:34.240321 20502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e317818ae3d45b78001389f1f44a695, State: Initialized, Role: FOLLOWER
I20260807 15:17:34.240821 20502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:34.240938 20502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:34.241031 20502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:34.241164 20502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:17:34.243333 20502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:34.243682 20502 leader_election.cc:302] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e317818ae3d45b78001389f1f44a695; no voters:
I20260807 15:17:34.244062 20502 leader_election.cc:288] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:17:34.244143 20509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:17:34.244345 20509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 LEADER]: Becoming Leader. State: Replica: 4e317818ae3d45b78001389f1f44a695, State: Running, Role: LEADER
I20260807 15:17:34.244652 20509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:34.245361 20502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:34.246029 20510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4e317818ae3d45b78001389f1f44a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } } }
I20260807 15:17:34.246158 20510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:34.246824 20517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:34.246857 20511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e317818ae3d45b78001389f1f44a695. Latest consensus state: current_term: 2 leader_uuid: "4e317818ae3d45b78001389f1f44a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } } }
I20260807 15:17:34.246969 20511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:34.251981 20517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:34.252710 20517 catalog_manager.cc:1259] Loaded cluster ID: 4439192a7ac24836881b1844e4fa5ae8
I20260807 15:17:34.252774 20517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:34.254138 20517 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 15:17:34.258553 20517 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 15:17:34.264240 20517 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:34.265058 20517 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Loaded TSK: 0
I20260807 15:17:34.266333 20517 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:17:34.345633 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:34.345988 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:34.346053 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:34.351886 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:34.351964 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:34.352053 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:17:34.359503 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:36501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=33613
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:34.360805 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:34.362155 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:34.374087 20541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.374034 20540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.374450 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:34.374620 20506 server_base.cc:1034] running on GCE node
I20260807 15:17:34.375018 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:34.375703 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:34.376895 20506 hybrid_clock.cc:648] HybridClock initialized: now 1786115854376864 us; error 46 us; skew 500 ppm
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:34.379925 20506 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:17:34.380913 20506 webserver.cc:460] Webserver started at http://127.31.230.193:33613/ using document root <none> and password file <none>
I20260807 15:17:34.381500 20506 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:34.381656 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:34.385763 20506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:17:34.388110 20558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.389469 20506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:34.389652 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "62f23dffbe5a438691c5bb9ba6d343b8"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "21d3880d80810ebd2b423b91c9c00636"
server_key_iv: "0723077cc08c62bbff2544709a8f6130"
server_key_version: "encryptionkey@0"
I20260807 15:17:34.390214 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:34.415791 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:34.419333 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:34.419564 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:34.420226 20506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:34.421579 20567 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:17:34.426105 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:17:34.426175 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260807 15:17:34.426265 20506 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:17:34.427983 20506 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:17:34.428046 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 15:17:34.428310 20567 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap starting.
I20260807 15:17:34.439664 20506 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:36501
I20260807 15:17:34.439692 20732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:36501 every 8 connection(s)
I20260807 15:17:34.440989 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:17:34.441318 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 20506
I20260807 15:17:34.441857 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:35395
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:45363
--webserver_port=39751
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:34.450580 20567 log.cc:824] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:34.458971 20567 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:34.459456 20567 tablet_bootstrap.cc:492] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap complete.
I20260807 15:17:34.460816 20567 ts_tablet_manager.cc:1397] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.008s
I20260807 15:17:34.462042 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.443622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:48145)
0807 15:17:34.444304 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:34.444366 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:34.445708 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:34.447065 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:34.447076 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:34.447604 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:34.448378 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.448404 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.450569 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.450572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.451596 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.451640 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.452787 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.452791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.452990 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.453777 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:34.453798 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:34.455789 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:34.458502 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.458508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.458520 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.458902 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.459306 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.459309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.459311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.459484 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.460200 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:34.460209 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:34.460519 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 15:17:34.460726 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 15:17:34.461062 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":168,"threads_started":1}
I20260807 15:17:34.462081 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.444083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:48145 (local address 127.31.230.254:45363)
0807 15:17:34.444524 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.444544 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.444590 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:17:34.446272 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.446284 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.446586 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.446859 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:34.448615 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.450376 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.451773 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.452605 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.452826 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.456025 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:34.456035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:34.456055 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:34.456101 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:34.458369 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.459032 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.459038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.459044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.459197 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.459623 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.459632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.459634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.460072 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:34.460208 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.460859 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.461224 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":114,"threads_started":1}
I20260807 15:17:34.463727 20567 raft_consensus.cc:348] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:34.464176 20567 raft_consensus.cc:729] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f23dffbe5a438691c5bb9ba6d343b8, State: Initialized, Role: FOLLOWER
I20260807 15:17:34.464690 20567 consensus_queue.cc:260] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } }
I20260807 15:17:34.464969 20737 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45363
I20260807 15:17:34.465216 20737 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:34.465737 20567 ts_tablet_manager.cc:1428] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260807 15:17:34.465797 20737 heartbeater.cc:511] Master 127.31.230.254:45363 requested a full tablet report, sending...
I20260807 15:17:34.468511 20438 ts_manager.cc:194] Registered new tserver with Master: 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501)
I20260807 15:17:34.472587 20438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:48145
I20260807 15:17:34.475256 20737 heartbeater.cc:503] Master 127.31.230.254:45363 was elected leader, sending a full tablet report...
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:17:34.483053 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.471934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36501 (local address 127.0.0.1:56906)
0807 15:17:34.472835 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:34.472886 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:34.472995 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:34.473755 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:34.473762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:34.474129 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:34.474231 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.474238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.475190 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.475193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.476141 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.476152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.476757 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.476762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.476880 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.477511 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:34.477588 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:34.479221 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:34.481152 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.481163 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.481169 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.481409 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.481789 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.481792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.481794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.481887 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.482366 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:34.482373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:34.482503 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:17:34.482648 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 15:17:34.482868 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":116,"threads_started":1}
I20260807 15:17:34.483059 20751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.471999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56906 (local address 127.31.230.193:36501)
0807 15:17:34.473215 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.473224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.473252 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:17:34.473340 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.473347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.473579 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.473739 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:34.474355 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.475069 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.476274 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.476614 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.476732 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.479361 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:34.479370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:34.479383 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:34.479416 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:34.481029 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.481564 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.481572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.481574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.481667 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.481986 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.481990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.481992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.482234 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:34.482391 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.482744 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.482911 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":1134,"thread_start_us":125,"threads_started":1}
I20260807 15:17:34.484071 20626 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5cbe58d5404a0881aeaa471866a68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:33 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:56906
I20260807 15:17:34.485227 20753 tablet_replica.cc:330] stopping tablet replica
I20260807 15:17:34.485580 20753 raft_consensus.cc:2229] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:17:34.485864 20753 raft_consensus.cc:2258] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:34.486864 20753 ts_tablet_manager.cc:1905] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:34.490590 20753 ts_tablet_manager.cc:1918] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:34.490715 20753 log.cc:1196] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/fe5cbe58d5404a0881aeaa471866a68d
I20260807 15:17:34.491021 20753 ts_tablet_manager.cc:1939] T fe5cbe58d5404a0881aeaa471866a68d P 62f23dffbe5a438691c5bb9ba6d343b8: Deleting consensus metadata
I20260807 15:17:34.492779 20417 catalog_manager.cc:4841] TS 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501): tablet fe5cbe58d5404a0881aeaa471866a68d (table test-table [id=b6b6b64725b5493bbe2c2d8d2fafdc1c]) successfully deleted
W20260807 15:17:34.570847 20742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:34.571098 20742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:34.571177 20742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:34.575371 20742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:34.575426 20742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:34.575547 20742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:17:34.580950 20742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:35395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=39751
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:34.582254 20742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:34.583310 20742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:34.593034 20760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.593043 20763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.593044 20758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:34.593638 20742 server_base.cc:1034] running on GCE node
I20260807 15:17:34.594056 20742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:34.594696 20742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:34.595878 20742 hybrid_clock.cc:648] HybridClock initialized: now 1786115854595836 us; error 46 us; skew 500 ppm
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:34.598763 20742 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:17:34.599642 20742 webserver.cc:460] Webserver started at http://127.31.230.194:39751/ using document root <none> and password file <none>
I20260807 15:17:34.600179 20742 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:34.600251 20742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:34.604146 20742 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:34.606204 20773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.607476 20742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:17:34.607614 20742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3b6d630c16ae4d6c9021875eff089c8a"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "f10f770a59ca97e166c766c19630fdee"
server_key_iv: "be4ee9a412b4c695f45e955cb98d34b6"
server_key_version: "encryptionkey@0"
I20260807 15:17:34.608193 20742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:34.629354 20742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:34.632862 20742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:34.633092 20742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:34.633867 20742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:34.634740 20742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:34.634812 20742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.634887 20742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:34.634931 20742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.644209 20742 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:35395
I20260807 15:17:34.644233 20935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:35395 every 8 connection(s)
I20260807 15:17:34.645258 20742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:17:34.652604 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 20742
I20260807 15:17:34.653069 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:41329
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:45363
--webserver_port=34895
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:34.660040 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.647345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:45383 (local address 127.31.230.254:45363)
0807 15:17:34.647511 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.647516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.648480 (+ 964us) server_negotiation.cc:407] Connection header received
0807 15:17:34.649985 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.649989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.650046 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.650126 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:34.651376 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.651921 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.653185 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.653567 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.653707 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.656294 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:34.656303 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:34.656305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:34.656331 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:34.657864 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.658391 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.658395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.658397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.658484 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.658843 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.658847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.658849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.659052 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:34.659165 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.659716 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.659860 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:17:34.660887 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.647846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:45383)
0807 15:17:34.648307 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:34.648351 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:34.649761 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:34.650347 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:34.650354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:34.650736 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:34.651238 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.651252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.652029 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.652031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.653035 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.653045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.653756 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.653760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.653942 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.654585 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:34.654601 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:34.656113 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:34.657983 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.657989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.658006 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.658281 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.658590 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.658593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.658595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.658743 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.659196 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:34.659203 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:34.659449 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 15:17:34.659673 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 15:17:34.660003 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":123,"threads_started":1}
I20260807 15:17:34.662194 20936 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45363
I20260807 15:17:34.662462 20936 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:34.662984 20936 heartbeater.cc:511] Master 127.31.230.254:45363 requested a full tablet report, sending...
I20260807 15:17:34.664127 20438 ts_manager.cc:194] Registered new tserver with Master: 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395)
I20260807 15:17:34.665037 20438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:45383
W20260807 15:17:34.771127 20942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:34.771359 20942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:34.771399 20942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:34.775564 20942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:34.775624 20942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:34.775682 20942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:17:34.781065 20942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:41329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=34895
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:34.782150 20942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:34.783121 20942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:34.793107 20955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.793169 20947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:34.793104 20951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:34.793443 20942 server_base.cc:1034] running on GCE node
I20260807 15:17:34.793890 20942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:34.794570 20942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:34.795737 20942 hybrid_clock.cc:648] HybridClock initialized: now 1786115854795723 us; error 24 us; skew 500 ppm
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:34.798614 20942 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:17:34.799530 20942 webserver.cc:460] Webserver started at http://127.31.230.195:34895/ using document root <none> and password file <none>
I20260807 15:17:34.800150 20942 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:34.800261 20942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:34.804298 20942 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 15:17:34.806456 20963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.807644 20942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:17:34.807794 20942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0bf29b062a904d6180416e3e6debd964"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "3013ef6839ce30d8ad8966bfa0c88cdb"
server_key_iv: "fe13c5c47282ee7a7ba9b6f730b6a5b7"
server_key_version: "encryptionkey@0"
I20260807 15:17:34.808346 20942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:34.828321 20942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:34.831959 20942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:34.832188 20942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:34.832825 20942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:34.833778 20942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:34.833850 20942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.833925 20942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:34.833968 20942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:34.843621 20942 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:41329
I20260807 15:17:34.843654 21129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:41329 every 8 connection(s)
I20260807 15:17:34.844650 20942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:17:34.851637 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 20942
Aug 07 15:17:34 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115854, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:34.859364 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.846738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:46759 (local address 127.31.230.254:45363)
0807 15:17:34.846895 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.846899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.847814 (+ 915us) server_negotiation.cc:407] Connection header received
0807 15:17:34.849217 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.849221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.849284 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.849399 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:34.850649 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.851206 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.852696 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.852929 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.853046 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.855585 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:34.855589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:34.855592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:34.855635 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:34.857156 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.857803 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.857808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.857810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.857899 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.858270 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.858273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.858275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.858449 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:34.858584 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.859082 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.859199 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:17:34.860307 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.852349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.31.230.254:45363)
0807 15:17:34.852626 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.852630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.852766 (+ 136us) server_negotiation.cc:407] Connection header received
0807 15:17:34.852939 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.852943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.852993 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.853090 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:34.854019 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.854534 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.855489 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.855722 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.855841 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.857078 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:34.857082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:34.857085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:34.857116 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:34.858609 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.859078 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.859087 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.859088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.859163 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:34.859450 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:34.859456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:34.859459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:34.859628 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:34.859841 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.860018 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.860136 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":111,"threads_started":1}
I20260807 15:17:34.860532 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.847194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:46759)
0807 15:17:34.847654 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:34.847697 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:34.848975 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:34.849642 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:34.849648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:34.850002 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:34.850489 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.850502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.851341 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.851352 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.852553 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.852567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.853088 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.853092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.853273 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.853921 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:34.853938 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:34.855416 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:34.857282 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.857288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.857303 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.857692 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.857997 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:34.858001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:34.858003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:34.858161 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:34.858561 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:34.858570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:34.858831 (+ 261us) client_negotiation.cc:769] Sending connection context
0807 15:17:34.859024 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 15:17:34.859361 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
I20260807 15:17:34.861696 21131 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45363
I20260807 15:17:34.861948 21131 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:34.862449 21131 heartbeater.cc:511] Master 127.31.230.254:45363 requested a full tablet report, sending...
I20260807 15:17:34.863463 20438 ts_manager.cc:194] Registered new tserver with Master: 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:34.863967 20438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:46759
I20260807 15:17:34.873957 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:17:34.878259 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.874538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.31.230.254:45363)
0807 15:17:34.874668 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.874672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.874830 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:17:34.874974 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.874976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.875072 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.875161 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:17:34.875387 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.875892 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.876762 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.877061 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.877145 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.877439 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:17:34.877940 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:17:34.878035 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.878105 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.878157 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:17:34.878901 20438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:17:34.882331 20438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:17:34.890401 20626 tablet_service.cc:1467] Processing CreateTablet for tablet e8b1820b6bd24edfb78064b9ae68f9d2 (DEFAULT_TABLE table=test-table [id=bc58795b27224a2f9941d5f1662401f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:34.891125 20626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8b1820b6bd24edfb78064b9ae68f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:34.894004 20567 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap starting.
I20260807 15:17:34.894991 20567 tablet_bootstrap.cc:654] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:34.895115 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.889092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35395 (local address 127.0.0.1:40156)
0807 15:17:34.889227 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:34.889246 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:34.889329 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:34.890245 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:34.890248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:34.890266 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:34.890379 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.890386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.892093 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.892098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.893392 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.893403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.894619 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.894623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.894725 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.894779 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:17:34.894862 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:17:34.894955 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1}
I20260807 15:17:34.895637 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.889328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.31.230.194:35395)
0807 15:17:34.889727 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.889740 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.889764 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:17:34.889854 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.889859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.890096 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.890275 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:34.890482 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.891958 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.893557 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.894481 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.894621 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.894762 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.895358 (+ 596us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.895468 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":156,"threads_started":1}
I20260807 15:17:34.895747 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.889350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41329 (local address 127.0.0.1:46836)
0807 15:17:34.889630 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:34.889644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:34.889738 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:34.890903 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:34.890906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:34.890923 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:34.891014 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.891021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.892362 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.892365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.893480 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:34.893491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.895265 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.895271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:34.895402 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.895435 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 15:17:34.895524 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:17:34.895599 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":17,"thread_start_us":133,"threads_started":1}
I20260807 15:17:34.895987 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:34.889915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.31.230.195:41329)
0807 15:17:34.890249 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:34.890261 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:34.890286 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:17:34.890365 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:34.890370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:34.890563 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:34.890721 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:34.891151 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.892211 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.893664 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:34.894925 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:34.895056 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:34.895188 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:34.895579 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 15:17:34.895687 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
I20260807 15:17:34.896809 20567 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: No bootstrap required, opened a new log
I20260807 15:17:34.896937 20567 ts_tablet_manager.cc:1397] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:34.897347 20567 raft_consensus.cc:348] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:34.897475 20567 raft_consensus.cc:374] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:34.897522 20567 raft_consensus.cc:729] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f23dffbe5a438691c5bb9ba6d343b8, State: Initialized, Role: FOLLOWER
I20260807 15:17:34.897701 20567 consensus_queue.cc:260] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:34.897996 20567 ts_tablet_manager.cc:1428] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:17:34.899114 21029 tablet_service.cc:1467] Processing CreateTablet for tablet e8b1820b6bd24edfb78064b9ae68f9d2 (DEFAULT_TABLE table=test-table [id=bc58795b27224a2f9941d5f1662401f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:34.899114 20828 tablet_service.cc:1467] Processing CreateTablet for tablet e8b1820b6bd24edfb78064b9ae68f9d2 (DEFAULT_TABLE table=test-table [id=bc58795b27224a2f9941d5f1662401f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:34.900280 20828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8b1820b6bd24edfb78064b9ae68f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:34.900291 21029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8b1820b6bd24edfb78064b9ae68f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:34.908798 21148 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Bootstrap starting.
I20260807 15:17:34.908798 21149 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Bootstrap starting.
I20260807 15:17:34.911891 21149 tablet_bootstrap.cc:654] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:34.911891 21148 tablet_bootstrap.cc:654] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:34.912855 21149 log.cc:824] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:34.912856 21148 log.cc:824] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:34.914809 21148 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: No bootstrap required, opened a new log
I20260807 15:17:34.914824 21149 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: No bootstrap required, opened a new log
I20260807 15:17:34.915025 21149 ts_tablet_manager.cc:1397] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:17:34.915024 21148 ts_tablet_manager.cc:1397] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:17:34.918256 21149 raft_consensus.cc:348] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:34.918283 21148 raft_consensus.cc:348] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:34.918502 21149 raft_consensus.cc:374] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:34.918542 21148 raft_consensus.cc:374] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:34.918597 21148 raft_consensus.cc:729] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6d630c16ae4d6c9021875eff089c8a, State: Initialized, Role: FOLLOWER
I20260807 15:17:34.918597 21149 raft_consensus.cc:729] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf29b062a904d6180416e3e6debd964, State: Initialized, Role: FOLLOWER
I20260807 15:17:34.919056 21149 consensus_queue.cc:260] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:34.919057 21148 consensus_queue.cc:260] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:34.919723 20936 heartbeater.cc:503] Master 127.31.230.254:45363 was elected leader, sending a full tablet report...
I20260807 15:17:34.919723 21131 heartbeater.cc:503] Master 127.31.230.254:45363 was elected leader, sending a full tablet report...
I20260807 15:17:34.920097 21149 ts_tablet_manager.cc:1428] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:17:34.920099 21148 ts_tablet_manager.cc:1428] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260807 15:17:34.942458 20738 tablet.cc:2369] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:17:35.052527 20748 raft_consensus.cc:482] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:17:35.052819 20748 raft_consensus.cc:504] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.054107 20748 leader_election.cc:288] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395), 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:35.058456 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.054533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:46671 (local address 127.31.230.194:35395)
0807 15:17:35.054689 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:35.054696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:35.054802 (+ 106us) server_negotiation.cc:407] Connection header received
0807 15:17:35.055062 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:35.055065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:35.055148 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:35.055194 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:35.055556 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.056159 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.057352 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.057818 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.057948 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:35.058007 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:35.058241 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 15:17:35.058316 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:17:35.058456 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.054969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:38379 (local address 127.31.230.195:41329)
0807 15:17:35.055122 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:35.055126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:35.055142 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:17:35.055231 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:35.055235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:35.055278 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:35.055374 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:35.055692 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.056201 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.057353 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.057842 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.057981 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:35.058031 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:35.058275 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 15:17:35.058347 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:17:35.058480 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.054341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35395 (local address 127.31.230.193:46671)
0807 15:17:35.054684 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.054707 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:35.054879 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:35.055324 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:35.055326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:35.055343 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:35.055448 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:35.055453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.056292 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.056298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:35.057214 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:35.057224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.057949 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.057952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:35.058035 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:35.058134 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 15:17:35.058198 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:17:35.058300 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":73,"thread_start_us":119,"threads_started":1}
I20260807 15:17:35.058495 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.054807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41329 (local address 127.31.230.193:38379)
0807 15:17:35.055019 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.055036 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:35.055139 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:35.055388 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:35.055391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:35.055409 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:35.055569 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:35.055575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.056359 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.056362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:35.057206 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:35.057215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.057997 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.058000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:35.058088 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:35.058142 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:17:35.058235 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:17:35.058316 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":64,"thread_start_us":60,"threads_started":1}
I20260807 15:17:35.059329 21063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8b1820b6bd24edfb78064b9ae68f9d2" candidate_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf29b062a904d6180416e3e6debd964" is_pre_election: true
I20260807 15:17:35.059329 20859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8b1820b6bd24edfb78064b9ae68f9d2" candidate_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b6d630c16ae4d6c9021875eff089c8a" is_pre_election: true
I20260807 15:17:35.059649 21063 raft_consensus.cc:2454] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62f23dffbe5a438691c5bb9ba6d343b8 in term 0.
I20260807 15:17:35.059664 20859 raft_consensus.cc:2454] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62f23dffbe5a438691c5bb9ba6d343b8 in term 0.
I20260807 15:17:35.060139 20564 leader_election.cc:302] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b6d630c16ae4d6c9021875eff089c8a, 62f23dffbe5a438691c5bb9ba6d343b8; no voters:
I20260807 15:17:35.060444 20748 raft_consensus.cc:2781] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:17:35.060545 20748 raft_consensus.cc:482] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:17:35.060611 20748 raft_consensus.cc:3037] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:35.061650 20748 raft_consensus.cc:504] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.062008 20748 leader_election.cc:288] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 1 election: Requested vote from peers 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395), 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:35.062400 20859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8b1820b6bd24edfb78064b9ae68f9d2" candidate_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b6d630c16ae4d6c9021875eff089c8a"
I20260807 15:17:35.062449 21063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8b1820b6bd24edfb78064b9ae68f9d2" candidate_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf29b062a904d6180416e3e6debd964"
I20260807 15:17:35.062510 20859 raft_consensus.cc:3037] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:35.062599 21063 raft_consensus.cc:3037] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:35.063629 21063 raft_consensus.cc:2454] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62f23dffbe5a438691c5bb9ba6d343b8 in term 1.
I20260807 15:17:35.063640 20859 raft_consensus.cc:2454] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62f23dffbe5a438691c5bb9ba6d343b8 in term 1.
I20260807 15:17:35.063905 20564 leader_election.cc:302] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf29b062a904d6180416e3e6debd964, 62f23dffbe5a438691c5bb9ba6d343b8; no voters:
I20260807 15:17:35.064088 20748 raft_consensus.cc:2781] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:35.064276 20748 raft_consensus.cc:686] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 LEADER]: Becoming Leader. State: Replica: 62f23dffbe5a438691c5bb9ba6d343b8, State: Running, Role: LEADER
I20260807 15:17:35.064539 20748 consensus_queue.cc:237] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.067642 20438 catalog_manager.cc:5495] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193). New cstate: current_term: 1 leader_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } health_report { overall_health: HEALTHY } } }
I20260807 15:17:35.091199 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.087753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.31.230.193:36501)
0807 15:17:35.088040 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:35.088047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:35.088060 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:17:35.088137 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:35.088139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:35.088192 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:35.088301 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:17:35.088508 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.089022 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.089887 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:35.090105 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:35.090188 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:35.090410 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:17:35.090851 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:17:35.090940 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:35.091024 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:17:35.091076 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":119,"threads_started":1}
I20260807 15:17:35.095402 20859 raft_consensus.cc:1261] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Refusing update from remote peer 62f23dffbe5a438691c5bb9ba6d343b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:35.095422 21063 raft_consensus.cc:1261] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Refusing update from remote peer 62f23dffbe5a438691c5bb9ba6d343b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 15:17:35.095767 21132 tablet.cc:2369] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:17:35.096246 21159 consensus_queue.cc:1035] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:35.096485 20748 consensus_queue.cc:1035] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:35.106680 21169 mvcc.cc:205] Tried to move back new op lower bound from 7315930542463442944 to 7315930542347120640. Current Snapshot: MvccSnapshot[applied={T|T < 7315930542463442944}]
I20260807 15:17:35.113790 20438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:35.114014 20438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:35.116760 20438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Sending DeleteTablet for 3 replicas of tablet e8b1820b6bd24edfb78064b9ae68f9d2
I20260807 15:17:35.117559 20828 tablet_service.cc:1514] Processing DeleteTablet for tablet e8b1820b6bd24edfb78064b9ae68f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:35 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:40156
I20260807 15:17:35.117635 20626 tablet_service.cc:1514] Processing DeleteTablet for tablet e8b1820b6bd24edfb78064b9ae68f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:35 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:56906
I20260807 15:17:35.117693 21029 tablet_service.cc:1514] Processing DeleteTablet for tablet e8b1820b6bd24edfb78064b9ae68f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:35 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:46836
I20260807 15:17:35.117825 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 20506
I20260807 15:17:35.120270 21184 tablet_replica.cc:330] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: stopping tablet replica
I20260807 15:17:35.120642 21184 raft_consensus.cc:2229] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:17:35.120963 21184 raft_consensus.cc:2258] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:35.123344 21184 ts_tablet_manager.cc:1905] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:35.125882 21184 ts_tablet_manager.cc:1918] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:35.126000 21184 log.cc:1196] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/e8b1820b6bd24edfb78064b9ae68f9d2
I20260807 15:17:35.126405 21184 ts_tablet_manager.cc:1939] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Deleting consensus metadata
I20260807 15:17:35.127696 20417 catalog_manager.cc:4841] TS 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329): tablet e8b1820b6bd24edfb78064b9ae68f9d2 (table test-table [id=bc58795b27224a2f9941d5f1662401f5]) successfully deleted
I20260807 15:17:35.133874 21183 tablet_replica.cc:330] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: stopping tablet replica
I20260807 15:17:35.134284 21183 raft_consensus.cc:2229] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:17:35.134660 21183 raft_consensus.cc:2258] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:35.136248 21183 ts_tablet_manager.cc:1905] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:17:35.136427 20417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:36501 (error 108)
I20260807 15:17:35.136768 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 20742
I20260807 15:17:35.139039 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.137302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36501 (local address 127.0.0.1:56924)
0807 15:17:35.138844 (+ 1542us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.138932 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1418}
W20260807 15:17:35.142376 20417 catalog_manager.cc:4584] TS 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8b1820b6bd24edfb78064b9ae68f9d2: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36501: connect: Connection refused (error 111)
I20260807 15:17:35.148231 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 20942
I20260807 15:17:35.148701 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.148351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35395 (local address 127.0.0.1:40168)
0807 15:17:35.148546 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.148605 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.194:35395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 15:17:35.156360 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 20388
I20260807 15:17:35.168704 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45363
--webserver_interface=127.31.230.254
--webserver_port=37427
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:35.288031 21186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:35.288301 21186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:35.288341 21186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:35.292443 21186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:35.292507 21186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:35.292527 21186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:35.292543 21186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:35.292558 21186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:35.298547 21186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:45363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=37427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:35.299640 21186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:35.300634 21186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:35.309531 21200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:35.309561 21193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:35.309567 21196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:35.309913 21186 server_base.cc:1034] running on GCE node
I20260807 15:17:35.310475 21186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:35.311640 21186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:35.312865 21186 hybrid_clock.cc:648] HybridClock initialized: now 1786115855312849 us; error 26 us; skew 500 ppm
Aug 07 15:17:35 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:35.315768 21186 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:35.316706 21186 webserver.cc:460] Webserver started at http://127.31.230.254:37427/ using document root <none> and password file <none>
I20260807 15:17:35.317294 21186 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:35.317359 21186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:35.321322 21186 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260807 15:17:35.323683 21208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:35.325079 21186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:35.325191 21186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "4e317818ae3d45b78001389f1f44a695"
format_stamp: "Formatted at 2026-08-07 15:17:32 on dist-test-slave-gkw7"
server_key: "6008723d28c89b9808ccc6dfefdd7c0b"
server_key_iv: "9667e5efd165169ceb94b637781aa291"
server_key_version: "encryptionkey@0"
I20260807 15:17:35.325811 21186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:35.352581 21186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:35.356253 21186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:35.356422 21186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:35.363731 21186 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:45363
I20260807 15:17:35.364032 21283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:45363 every 8 connection(s)
I20260807 15:17:35.364885 21186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:35.366410 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 21186
I20260807 15:17:35.366904 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:36501
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:45363
--webserver_port=33613
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:35.370296 21285 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:17:35.372004 21285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap starting.
I20260807 15:17:35.389966 21285 log.cc:824] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:35.401450 21285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:35.401937 21285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: Bootstrap complete.
I20260807 15:17:35.406080 21285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:35.406481 21285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e317818ae3d45b78001389f1f44a695, State: Initialized, Role: FOLLOWER
I20260807 15:17:35.407033 21285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:35.407155 21285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:35.407251 21285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:35.407368 21285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:17:35.409298 21285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:35.409685 21285 leader_election.cc:302] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e317818ae3d45b78001389f1f44a695; no voters:
I20260807 15:17:35.410102 21285 leader_election.cc:288] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:17:35.410192 21290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:17:35.410403 21290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [term 3 LEADER]: Becoming Leader. State: Replica: 4e317818ae3d45b78001389f1f44a695, State: Running, Role: LEADER
I20260807 15:17:35.410701 21290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } }
I20260807 15:17:35.411387 21285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:35.412122 21291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4e317818ae3d45b78001389f1f44a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } } }
I20260807 15:17:35.412299 21291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:35.412921 21292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e317818ae3d45b78001389f1f44a695. Latest consensus state: current_term: 3 leader_uuid: "4e317818ae3d45b78001389f1f44a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e317818ae3d45b78001389f1f44a695" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45363 } } }
I20260807 15:17:35.413201 21292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:35.416182 21309 catalog_manager.cc:1259] Loaded cluster ID: 4439192a7ac24836881b1844e4fa5ae8
I20260807 15:17:35.416271 21309 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: loading cluster ID for follower catalog manager: success
W20260807 15:17:35.417198 21309 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 4e317818ae3d45b78001389f1f44a695: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260807 15:17:35.417289 21309 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260807 15:17:35.417372 21302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:35.420135 21302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:35.420629 21302 catalog_manager.cc:1259] Loaded cluster ID: 4439192a7ac24836881b1844e4fa5ae8
I20260807 15:17:35.420706 21302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 15:17:35.421332 21302 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2bd0ba99ad google::LogMessage::Fail() at ??:0
@ 0x7f2bd0bac88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2bd0ba94de google::LogMessage::Flush() at ??:0
@ 0x7f2bd0baaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2bdb4e8fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f2bdb4e9d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2bdb4e6579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f2bdb52a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2bd2dfb3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2bd2dfbc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f2bd2dfd3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2bd2dec512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2bd44306db start_thread at ??:0
@ 0x7f2bcf6a761f clone at ??:0
W20260807 15:17:35.517848 21287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:35.518131 21287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:35.518178 21287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:35.522316 21287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:35.522382 21287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:35.522471 21287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:17:35.528134 21287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:36501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=33613
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:35.529465 21287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:35.530691 21287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:35.540798 21341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:35.540808 21342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:35.541174 21287 server_base.cc:1034] running on GCE node
W20260807 15:17:35.540863 21346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:35.541700 21287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:35.542413 21287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:35.543613 21287 hybrid_clock.cc:648] HybridClock initialized: now 1786115855543577 us; error 55 us; skew 500 ppm
Aug 07 15:17:35 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:35.546718 21287 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:17:35.547727 21287 webserver.cc:460] Webserver started at http://127.31.230.193:33613/ using document root <none> and password file <none>
I20260807 15:17:35.548314 21287 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:35.548377 21287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:35.552461 21287 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:17:35.554770 21356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:35.556110 21287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:17:35.556216 21287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "62f23dffbe5a438691c5bb9ba6d343b8"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "21d3880d80810ebd2b423b91c9c00636"
server_key_iv: "0723077cc08c62bbff2544709a8f6130"
server_key_version: "encryptionkey@0"
I20260807 15:17:35.556782 21287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:35.581703 21287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:35.585073 21287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:35.585305 21287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:35.586031 21287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:35.587212 21364 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:17:35.592726 21287 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:17:35.592850 21287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 15:17:35.592944 21287 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:17:35.594764 21287 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:17:35.594838 21287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 15:17:35.595155 21364 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap starting.
I20260807 15:17:35.606066 21287 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:36501
I20260807 15:17:35.606117 21521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:36501 every 8 connection(s)
I20260807 15:17:35.607075 21287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:17:35.616257 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 21287
I20260807 15:17:35.616725 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:35395
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:45363
--webserver_port=39751
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:17:35.617745 21364 log.cc:824] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:35.625272 21364 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:35.625751 21364 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Bootstrap complete.
I20260807 15:17:35.626955 21364 ts_tablet_manager.cc:1397] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.012s
I20260807 15:17:35.629710 21364 raft_consensus.cc:348] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.630156 21364 raft_consensus.cc:729] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f23dffbe5a438691c5bb9ba6d343b8, State: Initialized, Role: FOLLOWER
I20260807 15:17:35.630654 21364 consensus_queue.cc:260] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.631637 21364 ts_tablet_manager.cc:1428] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 15:17:35.647284 21525 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:17:35.609028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:38423)
0807 15:17:35.609550 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.609614 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:35.610429 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:35.646423 (+ 35994us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":155,"threads_started":1}
W20260807 15:17:35.647948 21523 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:45363 (0 consecutive failures): Network error: Failed to ping master at 127.31.230.254:45363: Client connection negotiation failed: client connection to 127.31.230.254:45363: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:17:35.648710 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.648444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:60113)
0807 15:17:35.648576 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.648654 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 15:17:35.649438 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.649224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:37561)
0807 15:17:35.649334 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.649380 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:17:35.649665 21523 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:17:35.741241 21529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:35.741503 21529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:35.741593 21529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:35.745743 21529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:35.745805 21529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:35.745893 21529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:17:35.751279 21529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:35395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=39751
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:35.752401 21529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:35.753470 21529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:35.763659 21536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:35.763751 21529 server_base.cc:1034] running on GCE node
W20260807 15:17:35.763665 21537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:35.763659 21539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:35.764324 21529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:35.765028 21529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:35.766219 21529 hybrid_clock.cc:648] HybridClock initialized: now 1786115855766186 us; error 52 us; skew 500 ppm
Aug 07 15:17:35 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:35.769826 21529 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:17:35.770782 21529 webserver.cc:460] Webserver started at http://127.31.230.194:39751/ using document root <none> and password file <none>
I20260807 15:17:35.771379 21529 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:35.771479 21529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:35.775609 21529 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:17:35.778038 21550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:35.779392 21529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:17:35.779533 21529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3b6d630c16ae4d6c9021875eff089c8a"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "f10f770a59ca97e166c766c19630fdee"
server_key_iv: "be4ee9a412b4c695f45e955cb98d34b6"
server_key_version: "encryptionkey@0"
I20260807 15:17:35.780090 21529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:35.802466 21529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:35.805751 21529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:35.805966 21529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:35.806607 21529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:35.807755 21559 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:17:35.812131 21529 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:17:35.812227 21529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 15:17:35.812350 21529 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:17:35.814067 21529 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:17:35.814143 21529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 15:17:35.814400 21559 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Bootstrap starting.
I20260807 15:17:35.820338 21559 log.cc:824] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:35.825423 21529 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:35395
I20260807 15:17:35.825482 21714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:35395 every 8 connection(s)
I20260807 15:17:35.826398 21529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:17:35.828959 21559 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:35.829450 21559 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Bootstrap complete.
I20260807 15:17:35.829929 21559 ts_tablet_manager.cc:1397] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.004s
I20260807 15:17:35.830041 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.828350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:59491)
0807 15:17:35.828888 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.829005 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":189,"threads_started":1}
W20260807 15:17:35.830476 21715 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:45363 (0 consecutive failures): Network error: Failed to ping master at 127.31.230.254:45363: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
I20260807 15:17:35.831116 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.830905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:42113)
0807 15:17:35.831016 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.831064 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 15:17:35.831779 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.831583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:55245)
0807 15:17:35.831685 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.831725 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 15:17:35.832101 21715 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:17:35.833320 21559 raft_consensus.cc:348] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.833810 21559 raft_consensus.cc:729] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6d630c16ae4d6c9021875eff089c8a, State: Initialized, Role: FOLLOWER
I20260807 15:17:35.834280 21559 consensus_queue.cc:260] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:35.835232 21559 ts_tablet_manager.cc:1428] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:17:35.835254 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.835067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:40949)
0807 15:17:35.835163 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.835206 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:17:35.835896 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:35.835674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:39847)
0807 15:17:35.835799 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:35.835848 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 15:17:35.836158 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 21529
I20260807 15:17:35.836573 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:41329
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:45363
--webserver_port=34895
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:35.953171 21719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:35.953497 21719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:35.953620 21719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:35.957625 21719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:35.957715 21719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:35.957814 21719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:17:35.963223 21719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:41329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=34895
--tserver_master_addrs=127.31.230.254:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:35.964354 21719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:35.965413 21719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:35.975685 21728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:35.975688 21730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:35.975684 21725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:35.976015 21719 server_base.cc:1034] running on GCE node
I20260807 15:17:35.976691 21719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:35.977421 21719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:35.978629 21719 hybrid_clock.cc:648] HybridClock initialized: now 1786115855978611 us; error 29 us; skew 500 ppm
Aug 07 15:17:35 dist-test-slave-gkw7 krb5kdc[19594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:35.981496 21719 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:17:35.982515 21719 webserver.cc:460] Webserver started at http://127.31.230.195:34895/ using document root <none> and password file <none>
I20260807 15:17:35.983158 21719 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:35.983256 21719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:35.987443 21719 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:17:35.989699 21742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:35.990870 21719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:17:35.991020 21719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "0bf29b062a904d6180416e3e6debd964"
format_stamp: "Formatted at 2026-08-07 15:17:33 on dist-test-slave-gkw7"
server_key: "3013ef6839ce30d8ad8966bfa0c88cdb"
server_key_iv: "fe13c5c47282ee7a7ba9b6f730b6a5b7"
server_key_version: "encryptionkey@0"
I20260807 15:17:35.991578 21719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:36.007313 21719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:36.010378 21719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:36.010592 21719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:36.011216 21719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:36.012063 21719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:36.012137 21719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:36.012238 21719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:36.012290 21719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:36.022208 21719 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:41329
I20260807 15:17:36.022228 21891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:41329 every 8 connection(s)
I20260807 15:17:36.023280 21719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:17:36.024395 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 21719
I20260807 15:17:36.027653 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:36.025770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:38633)
0807 15:17:36.026257 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:36.026385 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
W20260807 15:17:36.028101 21892 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:45363 (0 consecutive failures): Network error: Failed to ping master at 127.31.230.254:45363: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
I20260807 15:17:36.028728 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:36.028501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:39215)
0807 15:17:36.028622 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:36.028677 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 15:17:36.029412 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:36.029209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:32897)
0807 15:17:36.029323 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:36.029363 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:17:36.029624 21892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:17:36.650915 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:36.650405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:50913)
0807 15:17:36.650722 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:36.650791 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
I20260807 15:17:36.837188 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:36.836737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:51509)
0807 15:17:36.837013 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:36.837077 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260807 15:17:37.030740 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.030213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:47077)
0807 15:17:37.030537 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.030618 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":92,"threads_started":1}
I20260807 15:17:37.360657 21901 raft_consensus.cc:482] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:17:37.360881 21901 raft_consensus.cc:504] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:37.362121 21901 leader_election.cc:288] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395), 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
Aug 07 15:17:37 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.195@KRBTEST.COM
Aug 07 15:17:37 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.194@KRBTEST.COM
I20260807 15:17:37.379796 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.363049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:36807 (local address 127.31.230.195:41329)
0807 15:17:37.363381 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:37.363408 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:37.363440 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:17:37.364155 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:37.364163 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:37.364521 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:37.364829 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:37.365574 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.366555 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.367845 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.368281 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.368450 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:37.374898 (+ 6448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:37.374906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:37.374927 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:37.374962 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:37.376997 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:37.377657 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:37.377663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:37.377668 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:37.377799 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:37.378160 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:37.378164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:37.378166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:37.378599 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:37.378753 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:37.379305 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 15:17:37.379593 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260807 15:17:37.379783 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.362827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41329 (local address 127.31.230.193:36807)
0807 15:17:37.363101 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.363116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:37.363221 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:37.364793 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:37.364796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:37.364960 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:37.365344 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:37.365363 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.366727 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.366730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:37.367683 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:37.367693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.368409 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.368412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:37.368654 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:37.372793 (+ 4139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:37.372838 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:37.374611 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:37.377144 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:37.377150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:37.377166 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:37.377504 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:37.377900 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:37.377904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:37.377906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:37.378049 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:37.378726 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:37.378732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:37.378985 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 15:17:37.379184 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 15:17:37.379581 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":246,"spinlock_wait_cycles":46336,"thread_start_us":79,"threads_started":1}
I20260807 15:17:37.379854 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.362593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:35757 (local address 127.31.230.194:35395)
0807 15:17:37.363069 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:37.363086 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:37.363135 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:17:37.363436 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:37.363442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:37.363781 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:37.364041 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:37.365524 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.366558 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.367802 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.368284 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.368461 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:37.375161 (+ 6700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:37.375166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:37.375181 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:37.375211 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:37.377029 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:37.377684 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:37.377696 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:37.377701 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:37.377823 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:37.378156 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:37.378160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:37.378162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:37.378600 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:37.378753 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:37.379324 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 15:17:37.379621 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":82,"threads_started":1}
I20260807 15:17:37.379845 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.362434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35395 (local address 127.31.230.193:35757)
0807 15:17:37.362675 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.362690 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:37.362798 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:37.364171 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:37.364177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:37.364664 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:37.365343 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:37.365361 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.366719 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.366724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:37.367676 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:37.367686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:37.368432 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:37.368436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:37.368659 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:37.372797 (+ 4138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:37.372816 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:37.374973 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:37.377161 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:37.377165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:37.377167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:37.377512 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:37.377927 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:37.377931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:37.377933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:37.378052 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:37.378723 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:37.378730 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:37.378985 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 15:17:37.379183 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 15:17:37.379667 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":273,"spinlock_wait_cycles":37504,"thread_start_us":67,"threads_started":1}
I20260807 15:17:37.382275 21649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8b1820b6bd24edfb78064b9ae68f9d2" candidate_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3b6d630c16ae4d6c9021875eff089c8a" is_pre_election: true
I20260807 15:17:37.382649 21649 raft_consensus.cc:2454] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62f23dffbe5a438691c5bb9ba6d343b8 in term 1.
I20260807 15:17:37.383867 21361 leader_election.cc:302] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b6d630c16ae4d6c9021875eff089c8a, 62f23dffbe5a438691c5bb9ba6d343b8; no voters:
I20260807 15:17:37.384191 21901 raft_consensus.cc:2781] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:17:37.384343 21901 raft_consensus.cc:482] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:17:37.384409 21901 raft_consensus.cc:3037] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 1 FOLLOWER]: Advancing to term 2
W20260807 15:17:37.384857 21361 leader_election.cc:341] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329): Not found: Tablet not found: e8b1820b6bd24edfb78064b9ae68f9d2
I20260807 15:17:37.386848 21901 raft_consensus.cc:504] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:37.387218 21901 leader_election.cc:288] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 2 election: Requested vote from peers 3b6d630c16ae4d6c9021875eff089c8a (127.31.230.194:35395), 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329)
I20260807 15:17:37.388113 21649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8b1820b6bd24edfb78064b9ae68f9d2" candidate_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3b6d630c16ae4d6c9021875eff089c8a"
W20260807 15:17:37.388182 21361 leader_election.cc:341] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329): Not found: Tablet not found: e8b1820b6bd24edfb78064b9ae68f9d2
I20260807 15:17:37.388285 21649 raft_consensus.cc:3037] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:17:37.390640 21649 raft_consensus.cc:2454] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62f23dffbe5a438691c5bb9ba6d343b8 in term 2.
I20260807 15:17:37.390981 21361 leader_election.cc:302] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b6d630c16ae4d6c9021875eff089c8a, 62f23dffbe5a438691c5bb9ba6d343b8; no voters: 0bf29b062a904d6180416e3e6debd964
I20260807 15:17:37.391160 21901 raft_consensus.cc:2781] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:17:37.391423 21901 raft_consensus.cc:686] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [term 2 LEADER]: Becoming Leader. State: Replica: 62f23dffbe5a438691c5bb9ba6d343b8, State: Running, Role: LEADER
I20260807 15:17:37.391784 21901 consensus_queue.cc:237] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:37.392333 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.391951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:50735)
0807 15:17:37.392182 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.392229 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 15:17:37.838438 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.837970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:53853)
0807 15:17:37.838254 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.838325 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260807 15:17:37.846415 21649 raft_consensus.cc:1261] T e8b1820b6bd24edfb78064b9ae68f9d2 P 3b6d630c16ae4d6c9021875eff089c8a [term 2 FOLLOWER]: Refusing update from remote peer 62f23dffbe5a438691c5bb9ba6d343b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 15:17:37.847110 21901 consensus_queue.cc:1035] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 15:17:37.849349 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.849034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:54637)
0807 15:17:37.849178 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.849253 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:17:37.852001 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.851702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:46501)
0807 15:17:37.851863 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.851921 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 15:17:37.852208 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:37.851945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:42311)
0807 15:17:37.852086 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:37.852140 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260807 15:17:37.852499 21361 consensus_peers.cc:487] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8 -> Peer 0bf29b062a904d6180416e3e6debd964 (127.31.230.195:41329): Couldn't send request to peer 0bf29b062a904d6180416e3e6debd964. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8b1820b6bd24edfb78064b9ae68f9d2. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:17:38.032013 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.031577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:48595)
0807 15:17:38.031835 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.031896 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260807 15:17:38.302413 21918 ts_tablet_manager.cc:927] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Initiating tablet copy from peer 62f23dffbe5a438691c5bb9ba6d343b8 (127.31.230.193:36501)
I20260807 15:17:38.303131 21918 tablet_copy_client.cc:316] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: tablet copy: Beginning tablet copy session from remote peer at address 127.31.230.193:36501
Aug 07 15:17:38 dist-test-slave-gkw7 krb5kdc[19594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115855, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:17:38.313308 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.303698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36501 (local address 127.31.230.195:33779)
0807 15:17:38.303848 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.303878 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:38.304031 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:38.304602 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:38.304608 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:38.305030 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:38.305148 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:38.305155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:38.306361 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:38.306365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:38.307108 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:38.307119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:38.307699 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:38.307701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:38.307782 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:38.308415 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:38.308428 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:38.309864 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:38.311792 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:38.311800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:38.311805 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:38.312030 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:38.312302 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:38.312304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:38.312305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:38.312388 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:38.312816 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:38.312821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:38.312928 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 15:17:38.313046 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:17:38.313182 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 15:17:38.313397 21922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.303863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:33779 (local address 127.31.230.193:36501)
0807 15:17:38.304160 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:38.304166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:38.304189 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:17:38.304234 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:38.304239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:38.304433 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:38.304593 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:38.305319 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:38.306200 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:38.307230 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:38.307588 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:38.307676 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:38.309998 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:38.310007 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:38.310019 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:38.310043 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:38.311684 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:38.312128 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:38.312132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:38.312134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:38.312210 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:38.312466 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:38.312468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:38.312469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:38.312699 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:38.312823 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:38.313133 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 15:17:38.313276 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":63,"threads_started":1}
I20260807 15:17:38.314426 21481 tablet_copy_service.cc:140] P 62f23dffbe5a438691c5bb9ba6d343b8: Received BeginTabletCopySession request for tablet e8b1820b6bd24edfb78064b9ae68f9d2 from peer 0bf29b062a904d6180416e3e6debd964 ({username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:33779)
I20260807 15:17:38.314616 21481 tablet_copy_service.cc:161] P 62f23dffbe5a438691c5bb9ba6d343b8: Beginning new tablet copy session on tablet e8b1820b6bd24edfb78064b9ae68f9d2 from peer 0bf29b062a904d6180416e3e6debd964 at {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:33779: session id = 0bf29b062a904d6180416e3e6debd964-e8b1820b6bd24edfb78064b9ae68f9d2
I20260807 15:17:38.315779 21481 tablet_copy_source_session.cc:205] T e8b1820b6bd24edfb78064b9ae68f9d2 P 62f23dffbe5a438691c5bb9ba6d343b8: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:17:38.320065 21918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8b1820b6bd24edfb78064b9ae68f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:38.327148 21918 tablet_copy_client.cc:791] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: tablet copy: Starting download of 0 data blocks...
I20260807 15:17:38.327400 21918 tablet_copy_client.cc:655] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: tablet copy: Starting download of 1 WAL segments...
I20260807 15:17:38.329777 21918 tablet_copy_client.cc:531] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:17:38.331825 21918 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Bootstrap starting.
I20260807 15:17:38.337790 21918 log.cc:824] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:38.344607 21918 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:17:38.344887 21918 tablet_bootstrap.cc:492] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Bootstrap complete.
I20260807 15:17:38.345170 21918 ts_tablet_manager.cc:1397] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260807 15:17:38.347442 21918 raft_consensus.cc:348] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:38.347695 21918 raft_consensus.cc:729] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf29b062a904d6180416e3e6debd964, State: Initialized, Role: FOLLOWER
I20260807 15:17:38.348126 21918 consensus_queue.cc:260] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b6d630c16ae4d6c9021875eff089c8a" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35395 } } peers { permanent_uuid: "0bf29b062a904d6180416e3e6debd964" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41329 } } peers { permanent_uuid: "62f23dffbe5a438691c5bb9ba6d343b8" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36501 } }
I20260807 15:17:38.349066 21918 ts_tablet_manager.cc:1428] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260807 15:17:38.349573 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.349224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:36487)
0807 15:17:38.349373 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.349465 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 15:17:38.349834 21481 tablet_copy_service.cc:342] P 62f23dffbe5a438691c5bb9ba6d343b8: Request end of tablet copy session 0bf29b062a904d6180416e3e6debd964-e8b1820b6bd24edfb78064b9ae68f9d2 received from {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:33779
I20260807 15:17:38.349977 21481 tablet_copy_service.cc:433] P 62f23dffbe5a438691c5bb9ba6d343b8: ending tablet copy session 0bf29b062a904d6180416e3e6debd964-e8b1820b6bd24edfb78064b9ae68f9d2 on tablet e8b1820b6bd24edfb78064b9ae68f9d2 with peer 0bf29b062a904d6180416e3e6debd964
I20260807 15:17:38.350354 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.350065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:55207)
0807 15:17:38.350227 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.350268 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 15:17:38.815102 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.814770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:35063)
0807 15:17:38.814967 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.815013 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":45,"threads_started":1}
I20260807 15:17:38.815153 21828 raft_consensus.cc:1203] T e8b1820b6bd24edfb78064b9ae68f9d2 P 0bf29b062a904d6180416e3e6debd964 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 15:17:38.816238 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.815945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:47661)
0807 15:17:38.816084 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.816152 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 15:17:38.853314 21930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:38.852889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:37893)
0807 15:17:38.853163 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:38.853220 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":90,"threads_started":1}
I20260807 15:17:39.816403 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:39.815976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:56365)
0807 15:17:39.816244 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:39.816302 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
I20260807 15:17:39.817335 21932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:39.816910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:39045)
0807 15:17:39.817184 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:39.817237 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":74,"threads_started":1}
I20260807 15:17:39.854661 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:39.854206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:52087)
0807 15:17:39.854498 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:39.854558 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":107,"threads_started":1}
I20260807 15:17:40.817869 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:40.817386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:57679)
0807 15:17:40.817707 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:40.817770 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":80,"threads_started":1}
I20260807 15:17:40.818619 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:40.818118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:48379)
0807 15:17:40.818469 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:40.818530 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":82,"threads_started":1}
I20260807 15:17:40.818607 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:40.818400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:46681)
0807 15:17:40.818508 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:40.818555 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:17:40.856160 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:40.855727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:41437)
0807 15:17:40.855995 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:40.856058 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260807 15:17:41.820011 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:41.819530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:50737)
0807 15:17:41.819819 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:41.819899 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
I20260807 15:17:41.820066 21940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:41.819678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:55621)
0807 15:17:41.819927 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:41.819982 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
I20260807 15:17:41.857502 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:41.857032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:59153)
0807 15:17:41.857343 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:41.857402 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":121,"threads_started":1}
I20260807 15:17:42.821352 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:42.820848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:37939)
0807 15:17:42.821168 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:42.821236 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":93,"threads_started":1}
I20260807 15:17:42.821352 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:42.820848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:45905)
0807 15:17:42.821168 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:42.821235 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":111,"threads_started":1}
I20260807 15:17:42.858881 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:42.858389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:46031)
0807 15:17:42.858698 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:42.858765 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
I20260807 15:17:43.822751 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:43.822301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:39371)
0807 15:17:43.822586 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:43.822652 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260807 15:17:43.822798 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:43.822292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:45279)
0807 15:17:43.822649 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:43.822706 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":98,"threads_started":1}
I20260807 15:17:43.860153 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:43.859692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:33595)
0807 15:17:43.859977 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:43.860051 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":80,"threads_started":1}
I20260807 15:17:44.824082 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:44.823608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:58301)
0807 15:17:44.823915 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:44.823979 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260807 15:17:44.824119 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:44.823614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:35083)
0807 15:17:44.823971 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:44.824030 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":107,"threads_started":1}
I20260807 15:17:44.861650 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:44.861047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:37933)
0807 15:17:44.861410 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:44.861492 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":143,"threads_started":1}
I20260807 15:17:45.825480 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:45.824933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:45203)
0807 15:17:45.825293 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:45.825366 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":101,"threads_started":1}
I20260807 15:17:45.825723 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:45.825114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:37399)
0807 15:17:45.825452 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:45.825585 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":171,"threads_started":1}
I20260807 15:17:45.863025 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:45.862541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:45125)
0807 15:17:45.862850 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:45.862918 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":88,"threads_started":1}
I20260807 15:17:46.827068 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:46.826535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:57475)
0807 15:17:46.826928 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:46.826981 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":112,"threads_started":1}
I20260807 15:17:46.827039 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:46.826561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:43419)
0807 15:17:46.826860 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:46.826926 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260807 15:17:46.864284 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:46.863845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:45707)
0807 15:17:46.864122 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:46.864184 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260807 15:17:47.828387 21961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:47.827939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:57509)
0807 15:17:47.828227 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:47.828296 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":103,"threads_started":1}
I20260807 15:17:47.828446 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:47.827954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:55739)
0807 15:17:47.828295 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:47.828353 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":91,"threads_started":1}
I20260807 15:17:47.865854 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:47.865282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:38955)
0807 15:17:47.865667 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:47.865745 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
I20260807 15:17:48.829850 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:48.829311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:33441)
0807 15:17:48.829650 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:48.829741 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":96,"threads_started":1}
I20260807 15:17:48.829927 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:48.829311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:57243)
0807 15:17:48.829717 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:48.829814 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":100,"threads_started":1}
I20260807 15:17:48.867076 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:48.866636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:53573)
0807 15:17:48.866896 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:48.866953 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260807 15:17:49.831278 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:49.830801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:53295)
0807 15:17:49.831078 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:49.831159 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":104,"threads_started":1}
I20260807 15:17:49.831524 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:49.830938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:36691)
0807 15:17:49.831246 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:49.831315 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":75,"threads_started":1}
I20260807 15:17:49.868445 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:49.867926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:38973)
0807 15:17:49.868270 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:49.868341 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":85,"threads_started":1}
I20260807 15:17:50.832800 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:50.832229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.195:42563)
0807 15:17:50.832591 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:50.832666 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":136,"threads_started":1}
I20260807 15:17:50.833077 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:50.832559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.193:52771)
0807 15:17:50.832797 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:50.832848 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260807 15:17:50.870047 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:50.869502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45363 (local address 127.31.230.194:52741)
0807 15:17:50.869852 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:50.869915 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:45363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":77,"threads_started":1}
I20260807 15:17:51.031672 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 21287
I20260807 15:17:51.049798 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 21529
I20260807 15:17:51.061473 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 21719
2026-08-07T15:17:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19710 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:17:51 dist-test-slave-gkw7 krb5kdc[21985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:17:51 dist-test-slave-gkw7 krb5kdc[21985](info): set up 2 sockets
Aug 07 15:17:51 dist-test-slave-gkw7 krb5kdc[21985](info): commencing operation
krb5kdc: starting...
W20260807 15:17:52.305929 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:17:52 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:17:52.447028 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44645
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:52.569177 22004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:52.569478 22004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:52.569520 22004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:52.573926 22004 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:17:52.574000 22004 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:17:52.574018 22004 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:17:52.574031 22004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:17:52.574086 22004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:52.574124 22004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:52.574147 22004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:17:52.574185 22004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:17:52.579805 22004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:44645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:52.580946 22004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:52.582078 22004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:17:52.591478 22004 server_base.cc:1034] running on GCE node
W20260807 15:17:52.591436 22010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:52.591450 22009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:52.591755 22012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:52.592339 22004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:52.593648 22004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:52.594822 22004 hybrid_clock.cc:648] HybridClock initialized: now 1786115872594802 us; error 34 us; skew 500 ppm
Aug 07 15:17:52 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115872, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:52.597877 22004 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:17:52.598824 22004 webserver.cc:460] Webserver started at http://127.31.230.254:42295/ using document root <none> and password file <none>
I20260807 15:17:52.599450 22004 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:52.599515 22004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:52.599747 22004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:52.602006 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "bc847d9902c042d4b10b3e0fb397fbd3"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "cf4549d2899735567656816fff535241"
server_key_iv: "cc492c6f9de2f2cb8972360d05070319"
server_key_version: "encryptionkey@0"
I20260807 15:17:52.602612 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "bc847d9902c042d4b10b3e0fb397fbd3"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "cf4549d2899735567656816fff535241"
server_key_iv: "cc492c6f9de2f2cb8972360d05070319"
server_key_version: "encryptionkey@0"
I20260807 15:17:52.606602 22004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:17:52.609105 22026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:52.610515 22004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:17:52.610673 22004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "bc847d9902c042d4b10b3e0fb397fbd3"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "cf4549d2899735567656816fff535241"
server_key_iv: "cc492c6f9de2f2cb8972360d05070319"
server_key_version: "encryptionkey@0"
I20260807 15:17:52.610795 22004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:52.640405 22004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:52.643894 22004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:52.644137 22004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:52.651727 22004 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:44645
I20260807 15:17:52.652019 22095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:44645 every 8 connection(s)
I20260807 15:17:52.652885 22004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:17:52.656001 22096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:52.656081 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 22004
I20260807 15:17:52.656212 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:17:52.656536 32667 external_mini_cluster.cc:1428] Setting key e56f63f8a3bd1f7c5c7cab45d579786b
I20260807 15:17:52.662879 22096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap starting.
I20260807 15:17:52.665755 22096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Neither blocks nor log segments found. Creating new log.
Aug 07 15:17:52 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:52.666839 22096 log.cc:824] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:52.669456 22096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: No bootstrap required, opened a new log
I20260807 15:17:52.672169 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:52.658115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.31.230.254:44645)
0807 15:17:52.658589 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:52.658605 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:52.658642 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:17:52.659523 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:52.659545 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:52.659896 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:52.660259 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:52.661110 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:52.662216 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:52.663150 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:52.663831 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:52.663998 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:52.666790 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:52.666797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:52.666814 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:52.666847 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:52.668851 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:52.669301 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:52.669307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:52.669313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:52.669456 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:52.669824 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:52.669827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:52.669829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:52.670352 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:52.670549 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:52.670781 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:17:52.671089 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":138,"threads_started":1}
I20260807 15:17:52.674173 22096 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:17:52.674474 22096 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:52.674579 22096 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc847d9902c042d4b10b3e0fb397fbd3, State: Initialized, Role: FOLLOWER
I20260807 15:17:52.675047 22096 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:17:52.675205 22096 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:17:52.675275 22096 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:17:52.675374 22096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:52.676326 22096 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:17:52.676683 22096 leader_election.cc:302] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc847d9902c042d4b10b3e0fb397fbd3; no voters:
I20260807 15:17:52.677042 22096 leader_election.cc:288] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:17:52.677181 22102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:52.677426 22102 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 LEADER]: Becoming Leader. State: Replica: bc847d9902c042d4b10b3e0fb397fbd3, State: Running, Role: LEADER
I20260807 15:17:52.677760 22102 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:17:52.678328 22096 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:17:52.678958 22102 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc847d9902c042d4b10b3e0fb397fbd3. Latest consensus state: current_term: 1 leader_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } } }
I20260807 15:17:52.679159 22102 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:52.679418 22104 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } } }
I20260807 15:17:52.679533 22104 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:17:52.679646 22109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:17:52.682578 22109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:17:52.688478 22109 catalog_manager.cc:1347] Generated new cluster ID: 047f008ad1f346eb8f31a1fe9ab97f2b
I20260807 15:17:52.688561 22109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:17:52.707146 22109 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:17:52.708415 22109 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:17:52.713385 22109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Generated new TSK 0
I20260807 15:17:52.714262 22109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:17:52.748974 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44645
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:52.864017 22130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:52.864281 22130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:52.864324 22130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:52.868319 22130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:52.868382 22130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:52.868465 22130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:17:52.873808 22130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:52.874914 22130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:52.875953 22130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:52.886030 22141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:52.886026 22137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:52.886050 22139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:52.886497 22130 server_base.cc:1034] running on GCE node
I20260807 15:17:52.886931 22130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:52.887595 22130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:52.888756 22130 hybrid_clock.cc:648] HybridClock initialized: now 1786115872888740 us; error 26 us; skew 500 ppm
Aug 07 15:17:52 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115872, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:52.892056 22130 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:17:52.892936 22130 webserver.cc:460] Webserver started at http://127.31.230.193:43899/ using document root <none> and password file <none>
I20260807 15:17:52.893640 22130 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:52.893733 22130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:52.893967 22130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:52.896281 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "05b1ece7ca9845b3987a8316b98a99b1"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "b6687c28264457f6552403708958cdec"
server_key_iv: "9c6b53916f5d5264df09e35851118a9a"
server_key_version: "encryptionkey@0"
I20260807 15:17:52.896895 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "05b1ece7ca9845b3987a8316b98a99b1"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "b6687c28264457f6552403708958cdec"
server_key_iv: "9c6b53916f5d5264df09e35851118a9a"
server_key_version: "encryptionkey@0"
I20260807 15:17:52.900759 22130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:17:52.903391 22153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:52.904641 22130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:17:52.904786 22130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "05b1ece7ca9845b3987a8316b98a99b1"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "b6687c28264457f6552403708958cdec"
server_key_iv: "9c6b53916f5d5264df09e35851118a9a"
server_key_version: "encryptionkey@0"
I20260807 15:17:52.904903 22130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:52.921672 22130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:52.924885 22130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:52.925102 22130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:52.925783 22130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:52.926792 22130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:52.926849 22130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:52.926898 22130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:52.926923 22130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:52.935547 22130 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:38993
I20260807 15:17:52.935618 22302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:38993 every 8 connection(s)
I20260807 15:17:52.936483 22130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:17:52.936784 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 22130
I20260807 15:17:52.936944 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:17:52.937282 32667 external_mini_cluster.cc:1428] Setting key 9c4256020c6e7ddc7f0e295aa372e7c6
Aug 07 15:17:52 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115872, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:17:52.951639 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:52.938519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:37199 (local address 127.31.230.254:44645)
0807 15:17:52.938673 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:52.938679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:52.940150 (+ 1471us) server_negotiation.cc:407] Connection header received
0807 15:17:52.941484 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:52.941488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:52.941605 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:52.941693 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:52.942913 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:52.943472 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:52.944736 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:52.944988 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:52.945098 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:52.947775 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:52.947778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:52.947781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:52.947817 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:52.949467 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:52.950013 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:52.950017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:52.950019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:52.950105 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:52.950464 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:52.950468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:52.950470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:52.950663 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:52.950783 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:52.951326 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 15:17:52.951465 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:17:52.952553 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:52.938995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:37199)
0807 15:17:52.939964 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:52.940025 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:52.941264 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:52.941882 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:52.941888 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:52.942269 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:52.942760 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:52.942775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:52.943602 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:52.943606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:52.944593 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:52.944603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:52.945158 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:52.945162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:52.945419 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:52.946116 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:52.946133 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:52.947599 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:52.949610 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:52.949616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:52.949633 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:52.949900 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:52.950206 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:52.950209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:52.950211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:52.950362 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:52.950784 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:52.950792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:52.951054 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 15:17:52.951284 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 15:17:52.951625 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":538,"threads_started":1}
I20260807 15:17:52.954161 22303 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44645
I20260807 15:17:52.954460 22303 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:52.954994 22303 heartbeater.cc:511] Master 127.31.230.254:44645 requested a full tablet report, sending...
I20260807 15:17:52.956763 22051 ts_manager.cc:194] Registered new tserver with Master: 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993)
I20260807 15:17:52.957973 22051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:37199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:17:52.968606 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44645
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:53.084230 22312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:53.084499 22312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:53.084542 22312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:53.088562 22312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:53.088624 22312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:53.088681 22312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:17:53.093770 22312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:53.094772 22312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:53.095851 22312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:53.105607 22319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:53.105620 22321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:53.105752 22312 server_base.cc:1034] running on GCE node
W20260807 15:17:53.105620 22323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:53.106683 22312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:53.107359 22312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:53.108528 22312 hybrid_clock.cc:648] HybridClock initialized: now 1786115873108509 us; error 36 us; skew 500 ppm
Aug 07 15:17:53 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:53.111699 22312 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:17:53.112627 22312 webserver.cc:460] Webserver started at http://127.31.230.194:36869/ using document root <none> and password file <none>
I20260807 15:17:53.113210 22312 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:53.113301 22312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:53.113605 22312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:53.115852 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "3f9a9b26381a4f0bb9571da086355eb9"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "1e8f65bcc24581c17c7438e1c4549229"
server_key_iv: "8d38409f9f99d1500c58cd1d5241cc5a"
server_key_version: "encryptionkey@0"
I20260807 15:17:53.116425 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "3f9a9b26381a4f0bb9571da086355eb9"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "1e8f65bcc24581c17c7438e1c4549229"
server_key_iv: "8d38409f9f99d1500c58cd1d5241cc5a"
server_key_version: "encryptionkey@0"
I20260807 15:17:53.120270 22312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 15:17:53.122779 22330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:53.124010 22312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:17:53.124171 22312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3f9a9b26381a4f0bb9571da086355eb9"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "1e8f65bcc24581c17c7438e1c4549229"
server_key_iv: "8d38409f9f99d1500c58cd1d5241cc5a"
server_key_version: "encryptionkey@0"
I20260807 15:17:53.124300 22312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:53.135219 22312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:53.138243 22312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:53.138447 22312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:53.139034 22312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:53.139869 22312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:53.139940 22312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:53.140009 22312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:53.140064 22312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:53.148942 22312 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:40489
I20260807 15:17:53.148967 22476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:40489 every 8 connection(s)
I20260807 15:17:53.149852 22312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:17:53.156139 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 22312
I20260807 15:17:53.156313 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:17:53.156641 32667 external_mini_cluster.cc:1428] Setting key 34a54f96e86fabeb565e12cbee7eb803
Aug 07 15:17:53 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:53.167397 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.151952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:37829 (local address 127.31.230.254:44645)
0807 15:17:53.152104 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.152108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.153027 (+ 919us) server_negotiation.cc:407] Connection header received
0807 15:17:53.154519 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.154522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.154579 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.154661 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:53.156059 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.156703 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.158392 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.158686 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.158784 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.162509 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:53.162515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:53.162517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:53.162555 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:53.164812 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:53.165461 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:53.165465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:53.165469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:53.165627 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:53.166040 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:53.166046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:53.166050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:53.166361 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:53.166485 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.167022 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.167183 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:17:53.168138 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.152308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:37829)
0807 15:17:53.152846 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.152896 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.154276 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.154810 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.154817 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.155228 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:53.155833 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.155869 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.156859 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.156865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.158260 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.158271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.158884 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.158892 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.159188 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.159915 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:53.159947 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:53.162299 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:53.164975 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:53.164980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:53.164995 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:53.165316 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:53.165760 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:53.165763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:53.165764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:53.165910 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:53.166488 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:53.166493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:53.166737 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.166891 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.167158 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":130,"threads_started":1}
I20260807 15:17:53.169413 22478 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44645
I20260807 15:17:53.169756 22478 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:17:53.170426 22478 heartbeater.cc:511] Master 127.31.230.254:44645 requested a full tablet report, sending...
I20260807 15:17:53.171748 22051 ts_manager.cc:194] Registered new tserver with Master: 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489)
I20260807 15:17:53.172451 22051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:37829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:17:53.191028 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44645
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:17:53.307602 22484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:17:53.307904 22484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:17:53.307966 22484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:17:53.312175 22484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:17:53.312237 22484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:17:53.312316 22484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:17:53.318384 22484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:17:53.319514 22484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:17:53.320604 22484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:17:53.331053 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:53.331012 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:17:53.331012 22489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:17:53.331249 22484 server_base.cc:1034] running on GCE node
I20260807 15:17:53.331722 22484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:17:53.332440 22484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:17:53.333611 22484 hybrid_clock.cc:648] HybridClock initialized: now 1786115873333579 us; error 40 us; skew 500 ppm
Aug 07 15:17:53 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:17:53.336486 22484 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:17:53.337322 22484 webserver.cc:460] Webserver started at http://127.31.230.195:44721/ using document root <none> and password file <none>
I20260807 15:17:53.337957 22484 fs_manager.cc:359] Metadata directory not provided
I20260807 15:17:53.338020 22484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:17:53.338192 22484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:17:53.340224 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "b8e617e6635749d21d850cdb203f2d35"
server_key_iv: "e77a4117547c9f94897511546a8712aa"
server_key_version: "encryptionkey@0"
I20260807 15:17:53.340824 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "b8e617e6635749d21d850cdb203f2d35"
server_key_iv: "e77a4117547c9f94897511546a8712aa"
server_key_version: "encryptionkey@0"
I20260807 15:17:53.344666 22484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:17:53.347122 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:53.348453 22484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:17:53.348580 22484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "b8e617e6635749d21d850cdb203f2d35"
server_key_iv: "e77a4117547c9f94897511546a8712aa"
server_key_version: "encryptionkey@0"
I20260807 15:17:53.348704 22484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:17:53.374418 22484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:17:53.378171 22484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:17:53.378427 22484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:17:53.379146 22484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:17:53.380193 22484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:17:53.380244 22484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:53.380306 22484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:17:53.380340 22484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:17:53.389353 22484 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:33331
I20260807 15:17:53.389426 22666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:33331 every 8 connection(s)
I20260807 15:17:53.390333 22484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:17:53.398728 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 22484
I20260807 15:17:53.398875 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:17:53.399217 32667 external_mini_cluster.cc:1428] Setting key 92cc3dcc497d63f837af26f10a15071f
Aug 07 15:17:53 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:17:53.405753 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.392264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:34755 (local address 127.31.230.254:44645)
0807 15:17:53.392405 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.392409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.393217 (+ 808us) server_negotiation.cc:407] Connection header received
0807 15:17:53.394593 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.394596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.394644 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.394721 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:53.395972 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.396687 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.398102 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.398322 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.398423 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.401979 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:53.401982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:53.401984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:53.402010 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:53.403581 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:53.404135 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:53.404139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:53.404140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:53.404218 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:53.404584 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:53.404587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:53.404589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:53.404790 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:53.404902 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.405430 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.405581 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:17:53.406577 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.392568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:34755)
0807 15:17:53.393026 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.393069 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.394388 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.394885 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.394891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.395329 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:17:53.395813 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.395827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.396821 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.396824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.397993 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.398004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.398536 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.398542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.398773 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.399513 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:17:53.399533 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:17:53.401791 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:17:53.403715 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:53.403722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:53.403739 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:53.404012 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:53.404317 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:17:53.404320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:17:53.404322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:17:53.404474 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:17:53.404903 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:17:53.404911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:17:53.405170 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.405369 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.405738 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260807 15:17:53.407716 22667 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44645
I20260807 15:17:53.408018 22667 heartbeater.cc:464] Registering TS with master...
I20260807 15:17:53.408567 22667 heartbeater.cc:511] Master 127.31.230.254:44645 requested a full tablet report, sending...
I20260807 15:17:53.409794 22051 ts_manager.cc:194] Registered new tserver with Master: aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
I20260807 15:17:53.410415 22051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:34755
I20260807 15:17:53.413949 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:17:53.423837 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.416146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.31.230.254:44645)
0807 15:17:53.416327 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.416333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.416517 (+ 184us) server_negotiation.cc:407] Connection header received
0807 15:17:53.416697 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.416701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.416753 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.416840 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:17:53.417612 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.418163 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.419021 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.419251 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.419359 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.420739 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:17:53.420743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:17:53.420746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:17:53.420774 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:17:53.422045 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:53.422479 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:53.422483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:53.422485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:53.422569 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:17:53.422856 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:17:53.422860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:17:53.422862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:17:53.423088 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:17:53.423232 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.423501 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.423640 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 15:17:53.426823 22051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:17:53.430158 22051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:17:53.445340 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.440028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.31.230.195:33331)
0807 15:17:53.440584 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.440595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.440883 (+ 288us) server_negotiation.cc:407] Connection header received
0807 15:17:53.440968 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.440972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.441140 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.441279 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:53.441469 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.442614 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.444020 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.444609 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.444758 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.444901 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.445072 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.445175 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":174,"threads_started":1}
I20260807 15:17:53.445508 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.438924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.0.0.1:33320)
0807 15:17:53.439433 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.439482 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.439631 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.440974 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.440977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.440989 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:53.441107 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.441115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.442660 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.442664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.443966 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.443975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.445083 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.445087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.445195 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.445218 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.445273 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.445345 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":114,"thread_start_us":92,"threads_started":1}
I20260807 15:17:53.445690 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.439122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:40489 (local address 127.0.0.1:46324)
0807 15:17:53.439637 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.439653 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.439758 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.440908 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.440915 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.440944 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:53.441111 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.441116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.442323 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.442327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.443933 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.443946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.445272 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.445275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.445355 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.445386 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.445452 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.445510 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":145,"thread_start_us":123,"threads_started":1}
I20260807 15:17:53.445914 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.439249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.31.230.193:38993)
0807 15:17:53.440481 (+ 1232us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.440497 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.440520 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:17:53.440599 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.440606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.440811 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.440995 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:53.441245 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.442512 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.444097 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.444942 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.445458 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.445621 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.445671 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.445773 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1119,"thread_start_us":83,"threads_started":1}
I20260807 15:17:53.445330 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.439702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:33331 (local address 127.0.0.1:43596)
0807 15:17:53.440377 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.440391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.440786 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.441254 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.441259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.441276 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:53.441370 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.441377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.442727 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.442731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.443883 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.443893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.444745 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.444749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.444855 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.444890 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.445048 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.445176 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":5,"thread_start_us":125,"threads_started":1}
I20260807 15:17:53.446354 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.439781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.31.230.194:40489)
0807 15:17:53.440192 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.440202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.440224 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:17:53.440330 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.440337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.440566 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.440772 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:53.441279 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.442178 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.444095 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.444943 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.445078 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.445209 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.446080 (+ 871us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.446179 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":177,"threads_started":1}
I20260807 15:17:53.448999 22390 tablet_service.cc:1467] Processing CreateTablet for tablet a41e0c2bfad8464499b951e8cb59e073 (DEFAULT_TABLE table=test-table [id=fe636bc11a9e488091834dc83f45208c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:53.448997 22215 tablet_service.cc:1467] Processing CreateTablet for tablet a41e0c2bfad8464499b951e8cb59e073 (DEFAULT_TABLE table=test-table [id=fe636bc11a9e488091834dc83f45208c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:53.450084 22390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a41e0c2bfad8464499b951e8cb59e073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:53.450098 22215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a41e0c2bfad8464499b951e8cb59e073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:53.451658 22563 tablet_service.cc:1467] Processing CreateTablet for tablet a41e0c2bfad8464499b951e8cb59e073 (DEFAULT_TABLE table=test-table [id=fe636bc11a9e488091834dc83f45208c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:17:53.452735 22563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a41e0c2bfad8464499b951e8cb59e073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:17:53.457314 22688 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Bootstrap starting.
I20260807 15:17:53.458760 22689 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap starting.
I20260807 15:17:53.459836 22690 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Bootstrap starting.
I20260807 15:17:53.460335 22688 tablet_bootstrap.cc:654] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:53.461293 22688 log.cc:824] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:53.461383 22689 tablet_bootstrap.cc:654] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:53.462268 22690 tablet_bootstrap.cc:654] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Neither blocks nor log segments found. Creating new log.
I20260807 15:17:53.462257 22689 log.cc:824] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:53.463058 22690 log.cc:824] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:17:53.463887 22688 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: No bootstrap required, opened a new log
I20260807 15:17:53.464131 22688 ts_tablet_manager.cc:1397] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:17:53.464879 22690 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: No bootstrap required, opened a new log
I20260807 15:17:53.465067 22690 ts_tablet_manager.cc:1397] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:17:53.467300 22689 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: No bootstrap required, opened a new log
I20260807 15:17:53.467499 22689 ts_tablet_manager.cc:1397] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 15:17:53.467456 22688 raft_consensus.cc:348] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.467784 22688 raft_consensus.cc:374] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:53.467872 22688 raft_consensus.cc:729] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9a9b26381a4f0bb9571da086355eb9, State: Initialized, Role: FOLLOWER
I20260807 15:17:53.468423 22688 consensus_queue.cc:260] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.468716 22690 raft_consensus.cc:348] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.469090 22690 raft_consensus.cc:374] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:53.469108 22478 heartbeater.cc:503] Master 127.31.230.254:44645 was elected leader, sending a full tablet report...
I20260807 15:17:53.469233 22690 raft_consensus.cc:729] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa735e5a22ff43538e5f7d9f94a9bd3e, State: Initialized, Role: FOLLOWER
I20260807 15:17:53.469483 22688 ts_tablet_manager.cc:1428] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260807 15:17:53.469776 22690 consensus_queue.cc:260] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.470785 22690 ts_tablet_manager.cc:1428] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:17:53.471163 22667 heartbeater.cc:503] Master 127.31.230.254:44645 was elected leader, sending a full tablet report...
I20260807 15:17:53.471982 22689 raft_consensus.cc:348] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.472304 22689 raft_consensus.cc:374] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:17:53.472425 22689 raft_consensus.cc:729] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b1ece7ca9845b3987a8316b98a99b1, State: Initialized, Role: FOLLOWER
I20260807 15:17:53.472846 22689 consensus_queue.cc:260] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.473578 22303 heartbeater.cc:503] Master 127.31.230.254:44645 was elected leader, sending a full tablet report...
I20260807 15:17:53.473879 22689 ts_tablet_manager.cc:1428] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:17:53.505018 22700 raft_consensus.cc:482] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:17:53.505326 22700 raft_consensus.cc:504] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.506661 22700 leader_election.cc:288] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993), 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489)
I20260807 15:17:53.511070 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.507068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:50863 (local address 127.31.230.194:40489)
0807 15:17:53.507201 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.507206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.507459 (+ 253us) server_negotiation.cc:407] Connection header received
0807 15:17:53.507580 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.507584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.507640 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.507717 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:53.508090 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.508809 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.510085 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.510496 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.510628 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.510696 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.510877 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.510945 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:17:53.511178 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.507029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:34131 (local address 127.31.230.193:38993)
0807 15:17:53.507166 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.507170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.507187 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:17:53.507334 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.507337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.507384 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.507496 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:17:53.507852 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.508711 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.510107 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.510523 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.510671 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.510727 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.510912 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.510986 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:17:53.512274 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.506941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:40489 (local address 127.31.230.195:50863)
0807 15:17:53.507356 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.507367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.507450 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.507756 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.507761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.507782 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:53.507944 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.507951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.508929 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.508934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.509953 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.509965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.510626 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.510630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.510732 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.510760 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.510846 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.510923 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":22,"thread_start_us":151,"threads_started":1}
I20260807 15:17:53.512277 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.506855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.31.230.195:34131)
0807 15:17:53.507066 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.507082 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:17:53.507185 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:17:53.507521 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:17:53.507527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:17:53.507547 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:17:53.507689 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.507719 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.508870 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.508874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.509963 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:17:53.509974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.510662 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.510665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:17:53.510785 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.510802 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 15:17:53.510895 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:17:53.510971 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":28}
I20260807 15:17:53.513013 22242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a41e0c2bfad8464499b951e8cb59e073" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1ece7ca9845b3987a8316b98a99b1" is_pre_election: true
I20260807 15:17:53.513011 22415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a41e0c2bfad8464499b951e8cb59e073" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9a9b26381a4f0bb9571da086355eb9" is_pre_election: true
I20260807 15:17:53.513351 22242 raft_consensus.cc:2454] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa735e5a22ff43538e5f7d9f94a9bd3e in term 0.
I20260807 15:17:53.513370 22415 raft_consensus.cc:2454] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa735e5a22ff43538e5f7d9f94a9bd3e in term 0.
I20260807 15:17:53.513940 22505 leader_election.cc:302] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1, 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e; no voters:
I20260807 15:17:53.514278 22700 raft_consensus.cc:2781] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:17:53.514386 22700 raft_consensus.cc:482] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:17:53.514444 22700 raft_consensus.cc:3037] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:53.515771 22700 raft_consensus.cc:504] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.516145 22700 leader_election.cc:288] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 election: Requested vote from peers 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993), 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489)
I20260807 15:17:53.516573 22415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a41e0c2bfad8464499b951e8cb59e073" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9a9b26381a4f0bb9571da086355eb9"
I20260807 15:17:53.516592 22242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a41e0c2bfad8464499b951e8cb59e073" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1ece7ca9845b3987a8316b98a99b1"
I20260807 15:17:53.516721 22415 raft_consensus.cc:3037] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:53.516721 22242 raft_consensus.cc:3037] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:17:53.518155 22242 raft_consensus.cc:2454] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa735e5a22ff43538e5f7d9f94a9bd3e in term 1.
I20260807 15:17:53.518155 22415 raft_consensus.cc:2454] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa735e5a22ff43538e5f7d9f94a9bd3e in term 1.
I20260807 15:17:53.518476 22505 leader_election.cc:302] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e; no voters:
I20260807 15:17:53.518756 22700 raft_consensus.cc:2781] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:17:53.518999 22700 raft_consensus.cc:686] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 LEADER]: Becoming Leader. State: Replica: aa735e5a22ff43538e5f7d9f94a9bd3e, State: Running, Role: LEADER
I20260807 15:17:53.519348 22700 consensus_queue.cc:237] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:17:53.522912 22050 catalog_manager.cc:5495] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e reported cstate change: term changed from 0 to 1, leader changed from <none> to aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195). New cstate: current_term: 1 leader_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } health_report { overall_health: HEALTHY } } }
I20260807 15:17:53.549937 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.546198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.31.230.195:33331)
0807 15:17:53.546357 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:17:53.546362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:17:53.546378 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:17:53.546471 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:17:53.546474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:17:53.546530 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:17:53.546670 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:17:53.546931 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.547464 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.548356 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:17:53.548584 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:17:53.548697 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:17:53.549019 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:17:53.549455 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:17:53.549665 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 15:17:53.549727 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:17:53.549788 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:17:53.556293 22242 raft_consensus.cc:1261] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Refusing update from remote peer aa735e5a22ff43538e5f7d9f94a9bd3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:53.556293 22415 raft_consensus.cc:1261] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Refusing update from remote peer aa735e5a22ff43538e5f7d9f94a9bd3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:17:53.557135 22700 consensus_queue.cc:1035] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:53.557364 22704 consensus_queue.cc:1035] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:17:53.566807 22707 mvcc.cc:205] Tried to move back new op lower bound from 7315930618079272960 to 7315930617938186240. Current Snapshot: MvccSnapshot[applied={T|T < 7315930618079272960 or (T in {7315930618079272960})}]
I20260807 15:17:53.567240 22709 mvcc.cc:205] Tried to move back new op lower bound from 7315930618079272960 to 7315930617938186240. Current Snapshot: MvccSnapshot[applied={T|T < 7315930618079272960}]
I20260807 15:17:53.568845 22708 mvcc.cc:205] Tried to move back new op lower bound from 7315930618079272960 to 7315930617938186240. Current Snapshot: MvccSnapshot[applied={T|T < 7315930618079272960}]
I20260807 15:17:53.572762 22049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:53.572932 22049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:17:53.575402 22049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Sending DeleteTablet for 3 replicas of tablet a41e0c2bfad8464499b951e8cb59e073
I20260807 15:17:53.576202 22390 tablet_service.cc:1514] Processing DeleteTablet for tablet a41e0c2bfad8464499b951e8cb59e073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:46324
I20260807 15:17:53.576176 22215 tablet_service.cc:1514] Processing DeleteTablet for tablet a41e0c2bfad8464499b951e8cb59e073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:33320
I20260807 15:17:53.576704 22721 tablet_replica.cc:330] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: stopping tablet replica
I20260807 15:17:53.576701 22563 tablet_service.cc:1514] Processing DeleteTablet for tablet a41e0c2bfad8464499b951e8cb59e073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:43596
I20260807 15:17:53.576961 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 22130
I20260807 15:17:53.583328 22721 raft_consensus.cc:2229] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:17:53.583792 22721 raft_consensus.cc:2258] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:53.584645 22722 tablet_replica.cc:330] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: stopping tablet replica
I20260807 15:17:53.585033 22722 raft_consensus.cc:2229] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:17:53.585739 22722 raft_consensus.cc:2258] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:17:53.586500 22721 ts_tablet_manager.cc:1905] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:53.587277 22722 ts_tablet_manager.cc:1905] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:17:53.589548 22721 ts_tablet_manager.cc:1918] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:53.589663 22721 log.cc:1196] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/a41e0c2bfad8464499b951e8cb59e073
I20260807 15:17:53.590049 22721 ts_tablet_manager.cc:1939] T a41e0c2bfad8464499b951e8cb59e073 P 3f9a9b26381a4f0bb9571da086355eb9: Deleting consensus metadata
I20260807 15:17:53.590369 22722 ts_tablet_manager.cc:1918] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:17:53.590473 22722 log.cc:1196] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/a41e0c2bfad8464499b951e8cb59e073
I20260807 15:17:53.590837 22722 ts_tablet_manager.cc:1939] T a41e0c2bfad8464499b951e8cb59e073 P aa735e5a22ff43538e5f7d9f94a9bd3e: Deleting consensus metadata
I20260807 15:17:53.591645 22032 catalog_manager.cc:4841] TS 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): tablet a41e0c2bfad8464499b951e8cb59e073 (table test-table [id=fe636bc11a9e488091834dc83f45208c]) successfully deleted
I20260807 15:17:53.592181 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 22312
W20260807 15:17:53.592183 22028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:38993 (error 108)
I20260807 15:17:53.624182 22028 catalog_manager.cc:4841] TS aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): tablet a41e0c2bfad8464499b951e8cb59e073 (table test-table [id=fe636bc11a9e488091834dc83f45208c]) successfully deleted
I20260807 15:17:53.624985 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:17:53.624626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.0.0.1:33322)
0807 15:17:53.624802 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:17:53.624891 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 15:17:53.625284 22028 catalog_manager.cc:4584] TS 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a41e0c2bfad8464499b951e8cb59e073: Network error: Client connection negotiation failed: client connection to 127.31.230.193:38993: connect: Connection refused (error 111)
I20260807 15:17:53.630007 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 22484
I20260807 15:17:53.642632 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 22004
I20260807 15:18:13.653257 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44645
--webserver_interface=127.31.230.254
--webserver_port=42295
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:13.769929 22725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:13.770198 22725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:13.770255 22725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:13.774226 22725 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:18:13.774276 22725 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:18:13.774291 22725 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:18:13.774305 22725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:18:13.774323 22725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:13.774375 22725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:13.774410 22725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:18:13.774430 22725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:18:13.779649 22725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:44645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=42295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:13.780741 22725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:13.781879 22725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:18:13.797200 22725 server_base.cc:1034] running on GCE node
W20260807 15:18:13.797199 22738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:13.797199 22735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:13.797573 22740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:13.798280 22725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:13.799463 22725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:13.800642 22725 hybrid_clock.cc:648] HybridClock initialized: now 1786115893800622 us; error 32 us; skew 500 ppm
Aug 07 15:18:13 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115893, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:13.814150 22725 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:18:13.815320 22725 webserver.cc:460] Webserver started at http://127.31.230.254:42295/ using document root <none> and password file <none>
I20260807 15:18:13.816023 22725 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:13.816162 22725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:13.820605 22725 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:18:13.823109 22749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:13.824512 22725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:18:13.824690 22725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "bc847d9902c042d4b10b3e0fb397fbd3"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "cf4549d2899735567656816fff535241"
server_key_iv: "cc492c6f9de2f2cb8972360d05070319"
server_key_version: "encryptionkey@0"
I20260807 15:18:13.825420 22725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:13.848641 22725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:13.851841 22725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:13.852089 22725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:13.860244 22725 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:44645
I20260807 15:18:13.860548 22816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:44645 every 8 connection(s)
I20260807 15:18:13.861508 22725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:18:13.862003 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 22725
I20260807 15:18:13.862528 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:38993
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:44645
--webserver_port=43899
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:18:13.866918 22817 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:18:13.868705 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap starting.
I20260807 15:18:13.887215 22817 log.cc:824] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:13.897334 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:18:13.897848 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap complete.
I20260807 15:18:13.902459 22817 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:13.902916 22817 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc847d9902c042d4b10b3e0fb397fbd3, State: Initialized, Role: FOLLOWER
I20260807 15:18:13.903474 22817 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:13.903636 22817 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:18:13.903743 22817 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:18:13.903875 22817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:18:13.906005 22817 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:13.906371 22817 leader_election.cc:302] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc847d9902c042d4b10b3e0fb397fbd3; no voters:
I20260807 15:18:13.906774 22817 leader_election.cc:288] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:18:13.906867 22821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:18:13.907107 22821 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 LEADER]: Becoming Leader. State: Replica: bc847d9902c042d4b10b3e0fb397fbd3, State: Running, Role: LEADER
I20260807 15:18:13.907488 22821 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:13.908217 22817 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:18:13.909200 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc847d9902c042d4b10b3e0fb397fbd3. Latest consensus state: current_term: 2 leader_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } } }
I20260807 15:18:13.909250 22822 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } } }
I20260807 15:18:13.909358 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:13.909376 22822 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:13.909971 22833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:18:13.915998 22833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:18:13.916761 22833 catalog_manager.cc:1259] Loaded cluster ID: 047f008ad1f346eb8f31a1fe9ab97f2b
I20260807 15:18:13.916836 22833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:18:13.919610 22833 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:18:13.920394 22833 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Loaded TSK: 0
I20260807 15:18:13.944785 22833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Generated new TSK 1
I20260807 15:18:13.945087 22833 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:18:13.986761 22819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:13.987082 22819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:13.987156 22819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:13.991338 22819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:13.991426 22819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:13.991531 22819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:18:13.997841 22819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:38993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=43899
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:13.999030 22819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:14.000197 22819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:14.010645 22856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:14.010634 22858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:14.010936 22819 server_base.cc:1034] running on GCE node
W20260807 15:18:14.010628 22855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:14.011556 22819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:14.012266 22819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:14.013455 22819 hybrid_clock.cc:648] HybridClock initialized: now 1786115894013439 us; error 40 us; skew 500 ppm
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:14.016796 22819 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:18:14.017988 22819 webserver.cc:460] Webserver started at http://127.31.230.193:43899/ using document root <none> and password file <none>
I20260807 15:18:14.018735 22819 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:14.018857 22819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:14.024296 22819 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:18:14.027300 22866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.028905 22819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:18:14.029045 22819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "05b1ece7ca9845b3987a8316b98a99b1"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "b6687c28264457f6552403708958cdec"
server_key_iv: "9c6b53916f5d5264df09e35851118a9a"
server_key_version: "encryptionkey@0"
I20260807 15:18:14.029845 22819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:14.076416 22819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:14.079929 22819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:14.080153 22819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:14.080863 22819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:14.082204 22876 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:18:14.088178 22819 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:18:14.088279 22819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 15:18:14.088385 22819 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:18:14.090246 22819 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:18:14.090317 22819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 15:18:14.090622 22876 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap starting.
I20260807 15:18:14.101426 22819 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:38993
I20260807 15:18:14.101562 23032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:38993 every 8 connection(s)
I20260807 15:18:14.102468 22819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:18:14.111771 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 22819
I20260807 15:18:14.112216 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:40489
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:44645
--webserver_port=36869
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:18:14.113240 22876 log.cc:824] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:14.122040 22876 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:18:14.122531 22876 tablet_bootstrap.cc:492] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap complete.
I20260807 15:18:14.124006 22876 ts_tablet_manager.cc:1397] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.008s
I20260807 15:18:14.124266 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.104853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:45755)
0807 15:18:14.105520 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.105626 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.106891 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.108077 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.108086 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.108613 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:14.109448 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.109480 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.111021 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.111025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.112436 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.112447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.113511 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.113515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.113773 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.114782 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:14.114810 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:14.116923 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:14.120093 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.120099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.120111 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.120522 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.121034 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.121038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.121039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.121210 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.122275 (+ 1065us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:14.122281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:14.122599 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.122795 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.123175 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":159,"threads_started":1}
I20260807 15:18:14.124970 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.104870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:45755 (local address 127.31.230.254:44645)
0807 15:18:14.105309 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.105329 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.105807 (+ 478us) server_negotiation.cc:407] Connection header received
0807 15:18:14.107322 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.107331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.107630 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.107900 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:14.109679 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.110848 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.112559 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.113339 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.113516 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.117188 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:14.117199 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:14.117222 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:14.117264 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:14.119932 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.120689 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.120698 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.120708 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.120908 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.121352 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.121358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.121362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.122035 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:14.122216 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.123007 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.123409 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":160,"threads_started":1}
I20260807 15:18:14.126868 22876 raft_consensus.cc:348] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:18:14.127444 22876 raft_consensus.cc:729] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b1ece7ca9845b3987a8316b98a99b1, State: Initialized, Role: FOLLOWER
I20260807 15:18:14.127674 23033 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44645
I20260807 15:18:14.127956 23033 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:14.127939 22876 consensus_queue.cc:260] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } } peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } }
I20260807 15:18:14.128595 23033 heartbeater.cc:511] Master 127.31.230.254:44645 requested a full tablet report, sending...
I20260807 15:18:14.128821 22876 ts_tablet_manager.cc:1428] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:18:14.131601 22772 ts_manager.cc:194] Registered new tserver with Master: 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993)
I20260807 15:18:14.135083 22772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:45755
I20260807 15:18:14.137933 23033 heartbeater.cc:503] Master 127.31.230.254:44645 was elected leader, sending a full tablet report...
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115893, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.193@KRBTEST.COM
I20260807 15:18:14.146370 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.134752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.31.230.193:38993)
0807 15:18:14.135448 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.135462 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.136009 (+ 547us) server_negotiation.cc:407] Connection header received
0807 15:18:14.136108 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.136113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.136330 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.136479 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:14.137230 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.138623 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.139846 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.140206 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.140366 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.142683 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:14.142693 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:14.142706 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:14.142742 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:14.144367 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.144894 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.144899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.144900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.144977 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.145280 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.145284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.145285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.145523 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:14.145718 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.146062 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.146226 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":108,"threads_started":1}
I20260807 15:18:14.146370 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.134673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.0.0.1:55804)
0807 15:18:14.135700 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.135758 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.135912 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.136505 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.136513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.136961 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:14.137096 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.137104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.138758 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.138762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.139703 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.139713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.140365 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.140369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.140482 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.141079 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:14.141092 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:14.142543 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:14.144497 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.144506 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.144512 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.144754 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.145075 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.145080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.145082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.145175 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.145690 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:14.145696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:14.145828 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.145967 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.146171 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":124,"threads_started":1}
I20260807 15:18:14.147769 22932 tablet_service.cc:1514] Processing DeleteTablet for tablet a41e0c2bfad8464499b951e8cb59e073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:17:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:55804
I20260807 15:18:14.148672 23045 tablet_replica.cc:330] stopping tablet replica
I20260807 15:18:14.149024 23045 raft_consensus.cc:2229] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:14.149341 23045 raft_consensus.cc:2258] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:14.150568 23045 ts_tablet_manager.cc:1905] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:14.155550 23045 ts_tablet_manager.cc:1918] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:14.155689 23045 log.cc:1196] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal/wals/a41e0c2bfad8464499b951e8cb59e073
I20260807 15:18:14.156067 23045 ts_tablet_manager.cc:1939] T a41e0c2bfad8464499b951e8cb59e073 P 05b1ece7ca9845b3987a8316b98a99b1: Deleting consensus metadata
I20260807 15:18:14.158242 22751 catalog_manager.cc:4841] TS 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993): tablet a41e0c2bfad8464499b951e8cb59e073 (table test-table [id=fe636bc11a9e488091834dc83f45208c]) successfully deleted
W20260807 15:18:14.246301 23039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:14.246585 23039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:14.246632 23039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:14.250633 23039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:14.250692 23039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:14.250751 23039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:18:14.256168 23039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:40489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=36869
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:14.257408 23039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:14.258533 23039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:14.268410 23057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:14.268410 23060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:14.268749 23039 server_base.cc:1034] running on GCE node
W20260807 15:18:14.268445 23058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:14.269367 23039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:14.270084 23039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:14.271304 23039 hybrid_clock.cc:648] HybridClock initialized: now 1786115894271264 us; error 50 us; skew 500 ppm
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:14.274505 23039 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:18:14.275501 23039 webserver.cc:460] Webserver started at http://127.31.230.194:36869/ using document root <none> and password file <none>
I20260807 15:18:14.276147 23039 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:14.276279 23039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:14.280648 23039 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:18:14.282959 23069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.284273 23039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:18:14.284421 23039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3f9a9b26381a4f0bb9571da086355eb9"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "1e8f65bcc24581c17c7438e1c4549229"
server_key_iv: "8d38409f9f99d1500c58cd1d5241cc5a"
server_key_version: "encryptionkey@0"
I20260807 15:18:14.285020 23039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:14.298692 23039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:14.302109 23039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:14.302346 23039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:14.303015 23039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:14.304008 23039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:14.304096 23039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.304179 23039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:14.304240 23039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.314278 23039 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:40489
I20260807 15:18:14.314342 23228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:40489 every 8 connection(s)
I20260807 15:18:14.315539 23039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:18:14.322140 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 23039
I20260807 15:18:14.322690 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:33331
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:44645
--webserver_port=44721
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:14.331913 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.317463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:35793 (local address 127.31.230.254:44645)
0807 15:18:14.317672 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.317676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.318561 (+ 885us) server_negotiation.cc:407] Connection header received
0807 15:18:14.320079 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.320083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.320158 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.320250 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:14.321553 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.322177 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.324073 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.324334 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.324422 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.327217 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:14.327224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:14.327231 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:14.327269 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:14.329227 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.329898 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.329905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.329909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.330050 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.330443 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.330450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.330454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.330789 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:14.330917 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.331463 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.331645 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 15:18:14.332528 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.317912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:35793)
0807 15:18:14.318388 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.318433 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.319837 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.320445 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.320452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.320850 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:14.321364 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.321378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.322402 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.322408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.323914 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.323930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.324500 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.324507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.324752 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.325416 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:14.325434 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:14.327032 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:14.329366 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.329371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.329384 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.329785 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.330163 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.330166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.330167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.330324 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.330907 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:14.330912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:14.331171 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.331345 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.331607 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
I20260807 15:18:14.333827 23229 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44645
I20260807 15:18:14.334069 23229 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:14.334580 23229 heartbeater.cc:511] Master 127.31.230.254:44645 requested a full tablet report, sending...
I20260807 15:18:14.335775 22772 ts_manager.cc:194] Registered new tserver with Master: 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489)
I20260807 15:18:14.336421 22772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:35793
W20260807 15:18:14.443883 23233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:14.444304 23233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:14.444396 23233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:14.448632 23233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:14.448724 23233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:14.448812 23233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:18:14.454159 23233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:33331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=44721
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:14.455268 23233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:14.456343 23233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:14.466140 23243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:14.466147 23246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:14.466117 23241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:14.466414 23233 server_base.cc:1034] running on GCE node
I20260807 15:18:14.466868 23233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:14.467468 23233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:14.468644 23233 hybrid_clock.cc:648] HybridClock initialized: now 1786115894468609 us; error 45 us; skew 500 ppm
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:14.471494 23233 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:18:14.472515 23233 webserver.cc:460] Webserver started at http://127.31.230.195:44721/ using document root <none> and password file <none>
I20260807 15:18:14.473163 23233 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:14.473297 23233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:14.477644 23233 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:18:14.480257 23254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.481873 23233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:18:14.482033 23233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "b8e617e6635749d21d850cdb203f2d35"
server_key_iv: "e77a4117547c9f94897511546a8712aa"
server_key_version: "encryptionkey@0"
I20260807 15:18:14.482674 23233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:14.496356 23233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:14.499624 23233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:14.499851 23233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:14.500509 23233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:14.501438 23233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:14.501528 23233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.501662 23233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:14.501719 23233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:14.512342 23233 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:33331
I20260807 15:18:14.512370 23423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:33331 every 8 connection(s)
I20260807 15:18:14.513434 23233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:18:14.521421 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 23233
Aug 07 15:18:14 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115894, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:14.528995 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.515517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:56245 (local address 127.31.230.254:44645)
0807 15:18:14.515673 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.515678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.516630 (+ 952us) server_negotiation.cc:407] Connection header received
0807 15:18:14.518061 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.518064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.518114 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.518188 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:14.519371 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.519914 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.521009 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.521223 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.521300 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.524554 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:14.524557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:14.524559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:14.524583 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:14.526302 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.527093 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.527100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.527102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.527209 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.527660 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.527665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.527667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.527885 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:14.528015 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.528640 (+ 625us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.528782 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:18:14.530139 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.515997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:56245)
0807 15:18:14.516472 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.516515 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.517855 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.518360 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.518368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.518748 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:14.519231 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.519245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.520023 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.520026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.520901 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.520910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.521385 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.521390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.521676 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.522460 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:14.522482 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:14.524364 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:14.526458 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.526466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.526485 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.526952 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.527328 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:14.527333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:14.527335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:14.527526 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:14.528029 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:14.528040 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:14.528339 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.528589 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.528977 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":135,"threads_started":1}
I20260807 15:18:14.531306 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.522308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.31.230.254:44645)
0807 15:18:14.522635 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.522639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.522814 (+ 175us) server_negotiation.cc:407] Connection header received
0807 15:18:14.522945 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.522948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.522988 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.523063 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:14.523897 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.524631 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.525807 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.526047 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.526161 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.527503 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:14.527507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:14.527509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:14.527542 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:14.529019 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.529769 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.529773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.529775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.529856 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.530146 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.530150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.530152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.530410 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:14.530656 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.531026 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.531142 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":108,"threads_started":1}
I20260807 15:18:14.531643 23425 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44645
I20260807 15:18:14.531942 23425 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:14.532673 23425 heartbeater.cc:511] Master 127.31.230.254:44645 requested a full tablet report, sending...
I20260807 15:18:14.533879 22772 ts_manager.cc:194] Registered new tserver with Master: aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
I20260807 15:18:14.534531 22772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:56245
I20260807 15:18:14.545709 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:18:14.550161 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.546463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.31.230.254:44645)
0807 15:18:14.546616 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.546621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.546824 (+ 203us) server_negotiation.cc:407] Connection header received
0807 15:18:14.547000 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.547002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.547115 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.547206 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:18:14.547510 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.548041 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.548979 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.549212 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.549302 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.549660 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:18:14.549822 (+ 162us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 15:18:14.550049 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39394: token expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 15:18:14.550280 23430 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39394: token expired
W20260807 15:18:14.550760 22670 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.230.254:44645)
I20260807 15:18:14.558511 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.551188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.31.230.254:44645)
0807 15:18:14.551313 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.551316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.551331 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:18:14.551454 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.551456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.551493 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.551556 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:14.552261 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.552723 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.553605 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.553821 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.553903 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.555308 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:14.555311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:14.555313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:14.555336 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:14.556872 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.557306 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.557309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.557310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.557386 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:14.557716 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:14.557718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:14.557720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:14.557905 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:14.558006 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.558223 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.558322 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:18:14.561192 22772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:18:14.564541 22772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:18:14.573652 22932 tablet_service.cc:1467] Processing CreateTablet for tablet 68db851cc87745aebf3c8924f8e88d33 (DEFAULT_TABLE table=test-table [id=2d59caa486264c1ea0097595d1699032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:14.574384 22932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68db851cc87745aebf3c8924f8e88d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:14.577123 22876 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap starting.
I20260807 15:18:14.578042 22876 tablet_bootstrap.cc:654] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:14.578989 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.572377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:40489 (local address 127.0.0.1:46334)
0807 15:18:14.572594 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.572611 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.572694 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.573784 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.573787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.573807 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:14.573957 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.573964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.575458 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.575461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.576602 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.576609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.578494 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.578499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.578604 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.578651 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.578740 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.578835 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":33}
I20260807 15:18:14.579142 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.572550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.31.230.194:40489)
0807 15:18:14.573275 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.573287 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.573311 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:18:14.573387 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.573392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.573617 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.574116 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:14.574309 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.575336 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.577672 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.578364 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.578509 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.578672 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.578827 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.578955 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":114,"threads_started":1}
I20260807 15:18:14.579545 22876 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: No bootstrap required, opened a new log
I20260807 15:18:14.579635 22876 ts_tablet_manager.cc:1397] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:18:14.579910 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.572624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:33331 (local address 127.0.0.1:37894)
0807 15:18:14.572888 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.572899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.573005 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.573728 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.573731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.573751 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:14.573957 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.573964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.575248 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.575255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.576583 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.576596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.579476 (+ 2880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.579480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.579601 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.579631 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.579685 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.579760 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":67,"thread_start_us":114,"threads_started":1}
I20260807 15:18:14.580078 22876 raft_consensus.cc:348] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.580205 22876 raft_consensus.cc:374] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:14.580256 22876 raft_consensus.cc:729] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b1ece7ca9845b3987a8316b98a99b1, State: Initialized, Role: FOLLOWER
I20260807 15:18:14.580386 22876 consensus_queue.cc:260] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.580641 22876 ts_tablet_manager.cc:1428] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:18:14.580698 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.572894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.31.230.195:33331)
0807 15:18:14.573195 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.573207 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.573239 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:18:14.573310 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.573315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.573495 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.573763 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:14.574121 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.575131 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.576742 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.579338 (+ 2596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.580226 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.580370 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.580430 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.580549 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":104,"threads_started":1}
I20260807 15:18:14.582629 23141 tablet_service.cc:1467] Processing CreateTablet for tablet 68db851cc87745aebf3c8924f8e88d33 (DEFAULT_TABLE table=test-table [id=2d59caa486264c1ea0097595d1699032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:14.583853 23141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68db851cc87745aebf3c8924f8e88d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:14.583772 23322 tablet_service.cc:1467] Processing CreateTablet for tablet 68db851cc87745aebf3c8924f8e88d33 (DEFAULT_TABLE table=test-table [id=2d59caa486264c1ea0097595d1699032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:14.584860 23322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68db851cc87745aebf3c8924f8e88d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:14.592572 23446 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Bootstrap starting.
I20260807 15:18:14.592622 23447 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Bootstrap starting.
I20260807 15:18:14.595819 23447 tablet_bootstrap.cc:654] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:14.595819 23446 tablet_bootstrap.cc:654] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:14.596796 23446 log.cc:824] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:14.596796 23447 log.cc:824] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:14.599102 23447 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: No bootstrap required, opened a new log
I20260807 15:18:14.599102 23446 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: No bootstrap required, opened a new log
I20260807 15:18:14.599318 23446 ts_tablet_manager.cc:1397] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:18:14.599318 23447 ts_tablet_manager.cc:1397] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
W20260807 15:18:14.603631 23034 tablet.cc:2369] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:18:14.603446 23447 raft_consensus.cc:348] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.603446 23446 raft_consensus.cc:348] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.603749 23447 raft_consensus.cc:374] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:14.603749 23446 raft_consensus.cc:374] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:14.603814 23446 raft_consensus.cc:729] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9a9b26381a4f0bb9571da086355eb9, State: Initialized, Role: FOLLOWER
I20260807 15:18:14.603845 23447 raft_consensus.cc:729] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa735e5a22ff43538e5f7d9f94a9bd3e, State: Initialized, Role: FOLLOWER
I20260807 15:18:14.604352 23447 consensus_queue.cc:260] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.604352 23446 consensus_queue.cc:260] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.605083 23425 heartbeater.cc:503] Master 127.31.230.254:44645 was elected leader, sending a full tablet report...
I20260807 15:18:14.605319 23229 heartbeater.cc:503] Master 127.31.230.254:44645 was elected leader, sending a full tablet report...
I20260807 15:18:14.605466 23446 ts_tablet_manager.cc:1428] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:18:14.605466 23447 ts_tablet_manager.cc:1428] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:18:14.667629 23450 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:14.667860 23450 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.669140 23450 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331), 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993)
I20260807 15:18:14.669798 23451 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:14.670095 23451 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.671958 23451 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993)
I20260807 15:18:14.674381 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.669461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:33331 (local address 127.31.230.194:59671)
0807 15:18:14.669640 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.669659 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.669772 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.670565 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.670568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.670587 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:14.670744 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.670752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.671729 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.671745 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.672964 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.672976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.673968 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.673972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.674079 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.674103 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.674176 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.674244 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 15:18:14.674877 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.669573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59671 (local address 127.31.230.195:33331)
0807 15:18:14.670166 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.670173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.670191 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:18:14.670249 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.670252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.670306 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.670398 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:14.670882 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.671593 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.673115 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.673744 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.673859 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.673908 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.674688 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.674765 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":434}
I20260807 15:18:14.675400 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.669841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.31.230.194:54595)
0807 15:18:14.670331 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.670349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.670436 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.671141 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.671145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.671165 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:14.671292 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.671298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.672459 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.672462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.673864 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.673876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.674960 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.674965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.675096 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.675121 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.675182 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.675258 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":173,"threads_started":1}
I20260807 15:18:14.675420 23352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "3f9a9b26381a4f0bb9571da086355eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" is_pre_election: true
I20260807 15:18:14.675663 23352 raft_consensus.cc:2454] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f9a9b26381a4f0bb9571da086355eb9 in term 0.
I20260807 15:18:14.675997 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.671577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.31.230.195:51481)
0807 15:18:14.671739 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.671754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.671828 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.672507 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.672510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.672526 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:14.672641 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.672647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.673698 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.673701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.674836 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.674845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.675628 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.675632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.675740 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.675768 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.675819 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.675885 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 15:18:14.676090 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.670529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:54595 (local address 127.31.230.193:38993)
0807 15:18:14.670848 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.670854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.670872 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:18:14.670931 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.670934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.670989 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.671119 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:14.671477 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.672308 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.674019 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.674790 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.675622 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.675784 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.675828 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.675905 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260807 15:18:14.676196 23071 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e; no voters:
I20260807 15:18:14.676558 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.671817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:51481 (local address 127.31.230.193:38993)
0807 15:18:14.672280 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.672284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.672298 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:18:14.672339 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.672342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.672380 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.672742 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:14.672822 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.673574 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.674994 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.675483 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.676267 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.676320 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.676349 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.676420 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":58,"threads_started":1}
I20260807 15:18:14.676542 23450 raft_consensus.cc:2781] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:18:14.676693 23450 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:18:14.676757 23450 raft_consensus.cc:3037] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:14.676574 22963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "3f9a9b26381a4f0bb9571da086355eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1ece7ca9845b3987a8316b98a99b1" is_pre_election: true
I20260807 15:18:14.677088 22963 raft_consensus.cc:2454] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f9a9b26381a4f0bb9571da086355eb9 in term 0.
I20260807 15:18:14.677112 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1ece7ca9845b3987a8316b98a99b1" is_pre_election: true
I20260807 15:18:14.677310 22965 raft_consensus.cc:2454] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa735e5a22ff43538e5f7d9f94a9bd3e in term 0.
I20260807 15:18:14.677696 23257 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1, aa735e5a22ff43538e5f7d9f94a9bd3e; no voters:
I20260807 15:18:14.677999 23451 raft_consensus.cc:2781] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:18:14.678120 23451 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:18:14.678198 23451 raft_consensus.cc:3037] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:14.678367 23450 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.678509 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.671970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:48203 (local address 127.31.230.194:40489)
0807 15:18:14.672080 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.672085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.672633 (+ 548us) server_negotiation.cc:407] Connection header received
0807 15:18:14.672814 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.672817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.672858 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.672909 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:14.673284 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.674059 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.677218 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.677807 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.677896 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.677936 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.678348 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.678410 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:18:14.678534 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.671581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:40489 (local address 127.31.230.195:48203)
0807 15:18:14.672523 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.672538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:14.672700 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:14.673029 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:14.673031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:14.673048 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:14.673163 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.673169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.674195 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.674198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.676922 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:14.676935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.678052 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.678055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:14.678167 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.678202 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 15:18:14.678330 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:18:14.678400 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":512,"thread_start_us":115,"threads_started":1}
I20260807 15:18:14.678848 23450 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [CANDIDATE]: Term 1 election: Requested vote from peers aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331), 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993)
I20260807 15:18:14.679159 23164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9a9b26381a4f0bb9571da086355eb9" is_pre_election: true
I20260807 15:18:14.679391 23164 raft_consensus.cc:2379] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa735e5a22ff43538e5f7d9f94a9bd3e in current term 1: Already voted for candidate 3f9a9b26381a4f0bb9571da086355eb9 in this term.
I20260807 15:18:14.679421 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "3f9a9b26381a4f0bb9571da086355eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1ece7ca9845b3987a8316b98a99b1"
I20260807 15:18:14.679445 23352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "3f9a9b26381a4f0bb9571da086355eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e"
I20260807 15:18:14.679517 22965 raft_consensus.cc:3037] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:14.679648 23451 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.679903 23352 raft_consensus.cc:2379] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f9a9b26381a4f0bb9571da086355eb9 in current term 1: Already voted for candidate aa735e5a22ff43538e5f7d9f94a9bd3e in this term.
I20260807 15:18:14.680008 23451 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 election: Requested vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993)
I20260807 15:18:14.680452 22963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1ece7ca9845b3987a8316b98a99b1"
I20260807 15:18:14.680506 23164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68db851cc87745aebf3c8924f8e88d33" candidate_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9a9b26381a4f0bb9571da086355eb9"
I20260807 15:18:14.680652 23164 raft_consensus.cc:2379] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aa735e5a22ff43538e5f7d9f94a9bd3e in current term 1: Already voted for candidate 3f9a9b26381a4f0bb9571da086355eb9 in this term.
I20260807 15:18:14.680760 22965 raft_consensus.cc:2454] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f9a9b26381a4f0bb9571da086355eb9 in term 1.
I20260807 15:18:14.681021 23257 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa735e5a22ff43538e5f7d9f94a9bd3e; no voters: 05b1ece7ca9845b3987a8316b98a99b1, 3f9a9b26381a4f0bb9571da086355eb9
I20260807 15:18:14.681062 23071 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1, 3f9a9b26381a4f0bb9571da086355eb9; no voters: aa735e5a22ff43538e5f7d9f94a9bd3e
I20260807 15:18:14.681203 23451 raft_consensus.cc:2726] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 15:18:14.681314 23450 raft_consensus.cc:2781] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:14.681607 23450 raft_consensus.cc:686] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 LEADER]: Becoming Leader. State: Replica: 3f9a9b26381a4f0bb9571da086355eb9, State: Running, Role: LEADER
I20260807 15:18:14.681978 23450 consensus_queue.cc:237] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:14.685391 22772 catalog_manager.cc:5495] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "3f9a9b26381a4f0bb9571da086355eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } health_report { overall_health: UNKNOWN } } }
I20260807 15:18:14.702773 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.699241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46338 (local address 127.31.230.194:40489)
0807 15:18:14.699414 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:14.699418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:14.699433 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:18:14.699518 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:14.699520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:14.699596 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:14.699710 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:18:14.699944 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.700525 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.701309 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:14.701584 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:14.701678 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:14.701948 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:18:14.702382 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:18:14.702475 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:14.702531 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:18:14.702591 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:18:14.709353 22965 raft_consensus.cc:1261] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Refusing update from remote peer 3f9a9b26381a4f0bb9571da086355eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:14.709448 23352 raft_consensus.cc:1261] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Refusing update from remote peer 3f9a9b26381a4f0bb9571da086355eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:14.710088 23450 consensus_queue.cc:1035] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:14.710312 23457 consensus_queue.cc:1035] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:14.715502 23460 mvcc.cc:205] Tried to move back new op lower bound from 7315930704721293312 to 7315930704620019712. Current Snapshot: MvccSnapshot[applied={T|T < 7315930704721293312}]
I20260807 15:18:14.726293 22770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:14.726502 22770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:14.729049 22770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Sending DeleteTablet for 3 replicas of tablet 68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:14.729890 23141 tablet_service.cc:1514] Processing DeleteTablet for tablet 68db851cc87745aebf3c8924f8e88d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:14 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:46334
I20260807 15:18:14.730016 22932 tablet_service.cc:1514] Processing DeleteTablet for tablet 68db851cc87745aebf3c8924f8e88d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:14 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:55804
I20260807 15:18:14.730113 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 22819
I20260807 15:18:14.736023 23475 tablet_replica.cc:330] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: stopping tablet replica
I20260807 15:18:14.736475 23475 raft_consensus.cc:2229] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:18:14.737021 23475 raft_consensus.cc:2258] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:14.738945 23475 ts_tablet_manager.cc:1905] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:14.742113 23475 ts_tablet_manager.cc:1918] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:14.742091 23322 tablet_service.cc:1514] Processing DeleteTablet for tablet 68db851cc87745aebf3c8924f8e88d33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:14 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:37894
I20260807 15:18:14.742307 23475 log.cc:1196] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:14.742682 23475 ts_tablet_manager.cc:1939] T 68db851cc87745aebf3c8924f8e88d33 P 3f9a9b26381a4f0bb9571da086355eb9: Deleting consensus metadata
I20260807 15:18:14.743274 23477 tablet_replica.cc:330] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: stopping tablet replica
I20260807 15:18:14.743671 23477 raft_consensus.cc:2229] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:14.744042 22755 catalog_manager.cc:4841] TS 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): tablet 68db851cc87745aebf3c8924f8e88d33 (table test-table [id=2d59caa486264c1ea0097595d1699032]) successfully deleted
I20260807 15:18:14.744095 23477 raft_consensus.cc:2258] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 15:18:14.744091 22751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:38993 (error 108)
I20260807 15:18:14.745332 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 23039
I20260807 15:18:14.748406 23477 ts_tablet_manager.cc:1905] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:14.750763 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:14.750467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38993 (local address 127.0.0.1:55820)
0807 15:18:14.750612 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:14.750688 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 15:18:14.750981 22751 catalog_manager.cc:4584] TS 05b1ece7ca9845b3987a8316b98a99b1 (127.31.230.193:38993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68db851cc87745aebf3c8924f8e88d33: Network error: Client connection negotiation failed: client connection to 127.31.230.193:38993: connect: Connection refused (error 111)
I20260807 15:18:14.751441 23477 ts_tablet_manager.cc:1918] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:14.751596 23477 log.cc:1196] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:14.751992 23477 ts_tablet_manager.cc:1939] T 68db851cc87745aebf3c8924f8e88d33 P aa735e5a22ff43538e5f7d9f94a9bd3e: Deleting consensus metadata
I20260807 15:18:14.753428 22751 catalog_manager.cc:4841] TS aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): tablet 68db851cc87745aebf3c8924f8e88d33 (table test-table [id=2d59caa486264c1ea0097595d1699032]) successfully deleted
I20260807 15:18:14.756647 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 23233
I20260807 15:18:14.772887 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 22725
I20260807 15:18:34.788273 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44645
--webserver_interface=127.31.230.254
--webserver_port=42295
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:34.904642 23479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:34.904951 23479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:34.905022 23479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:34.909113 23479 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:18:34.909201 23479 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:18:34.909226 23479 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:18:34.909246 23479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:18:34.909297 23479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:34.909353 23479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:34.909399 23479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:18:34.909433 23479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:18:34.914857 23479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:44645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=42295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:34.916049 23479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:34.917127 23479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:34.925988 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:34.926123 23479 server_base.cc:1034] running on GCE node
W20260807 15:18:34.925979 23491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:34.925992 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:34.926797 23479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:34.927908 23479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:34.929119 23479 hybrid_clock.cc:648] HybridClock initialized: now 1786115914929089 us; error 44 us; skew 500 ppm
Aug 07 15:18:34 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:34.935873 23479 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:18:34.936857 23479 webserver.cc:460] Webserver started at http://127.31.230.254:42295/ using document root <none> and password file <none>
I20260807 15:18:34.937456 23479 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:34.937568 23479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:34.941448 23479 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:18:34.943825 23511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:34.945144 23479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:18:34.945289 23479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "bc847d9902c042d4b10b3e0fb397fbd3"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "cf4549d2899735567656816fff535241"
server_key_iv: "cc492c6f9de2f2cb8972360d05070319"
server_key_version: "encryptionkey@0"
I20260807 15:18:34.945921 23479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:34.961661 23479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:34.964504 23479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:34.964720 23479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:34.972430 23479 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:44645
I20260807 15:18:34.972738 23577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:44645 every 8 connection(s)
I20260807 15:18:34.973584 23479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:18:34.976953 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 23479
I20260807 15:18:34.977521 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:38993
--local_ip_for_outbound_sockets=127.31.230.193
--tserver_master_addrs=127.31.230.254:44645
--webserver_port=43899
--webserver_interface=127.31.230.193
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:18:34.978523 23578 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:18:34.980769 23578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap starting.
I20260807 15:18:34.999110 23578 log.cc:824] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:35.010645 23578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:18:35.011078 23578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Bootstrap complete.
I20260807 15:18:35.015496 23578 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:35.015901 23578 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc847d9902c042d4b10b3e0fb397fbd3, State: Initialized, Role: FOLLOWER
I20260807 15:18:35.016455 23578 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:35.016579 23578 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:18:35.016662 23578 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:18:35.016777 23578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:18:35.018802 23578 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:35.019136 23578 leader_election.cc:302] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc847d9902c042d4b10b3e0fb397fbd3; no voters:
I20260807 15:18:35.019500 23578 leader_election.cc:288] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:18:35.019594 23584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:18:35.019862 23584 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [term 3 LEADER]: Becoming Leader. State: Replica: bc847d9902c042d4b10b3e0fb397fbd3, State: Running, Role: LEADER
I20260807 15:18:35.020226 23584 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } }
I20260807 15:18:35.020804 23578 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:18:35.021713 23585 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } } }
I20260807 15:18:35.021864 23585 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:35.022523 23586 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc847d9902c042d4b10b3e0fb397fbd3. Latest consensus state: current_term: 3 leader_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc847d9902c042d4b10b3e0fb397fbd3" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44645 } } }
I20260807 15:18:35.022675 23586 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:35.023069 23600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:18:35.027612 23600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:18:35.028184 23600 catalog_manager.cc:1259] Loaded cluster ID: 047f008ad1f346eb8f31a1fe9ab97f2b
I20260807 15:18:35.028276 23600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:18:35.030184 23600 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:18:35.030829 23600 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Loaded TSK: 0
I20260807 15:18:35.030922 23600 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc847d9902c042d4b10b3e0fb397fbd3: Loaded TSK: 1
F20260807 15:18:35.031529 23600 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff5597bc9ad google::LogMessage::Fail() at ??:0
@ 0x7ff5597bf88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff5597bc4de google::LogMessage::Flush() at ??:0
@ 0x7ff5597bda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff55c48018d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff55c478f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff5641217bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff5640fb99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff5640fc178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff56413ddf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff56415c82b std::function<>::operator()() at ??:0
@ 0x7ff5640fbb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff5640fcf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff5640f9579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff56413d640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff55ba0e3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff55ba0ec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff55ba103f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff55b9ff512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff55d0436db start_thread at ??:0
@ 0x7ff5582ba61f clone at ??:0
W20260807 15:18:35.125613 23582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:35.125862 23582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:35.125907 23582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:35.130400 23582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:35.130460 23582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:35.130522 23582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:18:35.136258 23582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:38993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=43899
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:35.137449 23582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:35.138577 23582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:35.148520 23645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:35.148591 23644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:35.148507 23649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:35.148864 23582 server_base.cc:1034] running on GCE node
I20260807 15:18:35.149259 23582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:35.151490 23582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:35.152685 23582 hybrid_clock.cc:648] HybridClock initialized: now 1786115915152658 us; error 46 us; skew 500 ppm
Aug 07 15:18:35 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115915, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:35.155669 23582 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:18:35.156610 23582 webserver.cc:460] Webserver started at http://127.31.230.193:43899/ using document root <none> and password file <none>
I20260807 15:18:35.157182 23582 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:35.157253 23582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:35.161468 23582 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:18:35.163865 23658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.165333 23582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:18:35.165462 23582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "05b1ece7ca9845b3987a8316b98a99b1"
format_stamp: "Formatted at 2026-08-07 15:17:52 on dist-test-slave-gkw7"
server_key: "b6687c28264457f6552403708958cdec"
server_key_iv: "9c6b53916f5d5264df09e35851118a9a"
server_key_version: "encryptionkey@0"
I20260807 15:18:35.166076 23582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:35.184252 23582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:35.187691 23582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:35.187907 23582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:35.188556 23582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:35.189738 23670 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:18:35.195286 23582 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:18:35.195365 23582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260807 15:18:35.195444 23582 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:18:35.197125 23582 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:18:35.197196 23582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 15:18:35.197582 23670 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap starting.
I20260807 15:18:35.208333 23582 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:38993
I20260807 15:18:35.208392 23821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:38993 every 8 connection(s)
I20260807 15:18:35.209345 23582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:18:35.216135 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 23582
I20260807 15:18:35.216651 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:40489
--local_ip_for_outbound_sockets=127.31.230.194
--tserver_master_addrs=127.31.230.254:44645
--webserver_port=36869
--webserver_interface=127.31.230.194
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:18:35.220289 23670 log.cc:824] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:35.230015 23670 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:18:35.230494 23670 tablet_bootstrap.cc:492] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Bootstrap complete.
I20260807 15:18:35.232039 23670 ts_tablet_manager.cc:1397] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent bootstrapping tablet: real 0.035s user 0.019s sys 0.013s
I20260807 15:18:35.235663 23670 raft_consensus.cc:348] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:35.236222 23670 raft_consensus.cc:729] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b1ece7ca9845b3987a8316b98a99b1, State: Initialized, Role: FOLLOWER
I20260807 15:18:35.236760 23670 consensus_queue.cc:260] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:35.237892 23670 ts_tablet_manager.cc:1428] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260807 15:18:35.292798 23824 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:18:35.211343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:39415)
0807 15:18:35.211868 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.211923 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:35.212752 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:35.292000 (+ 79248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":151,"threads_started":1}
W20260807 15:18:35.293325 23822 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:44645 (0 consecutive failures): Network error: Failed to ping master at 127.31.230.254:44645: Client connection negotiation failed: client connection to 127.31.230.254:44645: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:18:35.294174 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.293933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:48125)
0807 15:18:35.294056 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.294118 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:18:35.294884 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.294699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:49561)
0807 15:18:35.294796 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.294838 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 15:18:35.295112 23822 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:18:35.369253 23826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:35.369573 23826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:35.369669 23826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:35.374241 23826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:35.374301 23826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:35.374398 23826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:18:35.379674 23826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:40489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=36869
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:35.380738 23826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:35.381865 23826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:35.391916 23840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:35.391947 23838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:35.392066 23826 server_base.cc:1034] running on GCE node
W20260807 15:18:35.391961 23837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:35.392604 23826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:35.393283 23826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:35.394455 23826 hybrid_clock.cc:648] HybridClock initialized: now 1786115915394422 us; error 45 us; skew 500 ppm
Aug 07 15:18:35 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115915, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:35.397648 23826 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:18:35.398577 23826 webserver.cc:460] Webserver started at http://127.31.230.194:36869/ using document root <none> and password file <none>
I20260807 15:18:35.399204 23826 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:35.399295 23826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:35.403437 23826 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 15:18:35.405659 23854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.406898 23826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:18:35.407030 23826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "3f9a9b26381a4f0bb9571da086355eb9"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "1e8f65bcc24581c17c7438e1c4549229"
server_key_iv: "8d38409f9f99d1500c58cd1d5241cc5a"
server_key_version: "encryptionkey@0"
I20260807 15:18:35.407613 23826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:35.421360 23826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:35.424670 23826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:35.424880 23826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:35.425485 23826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:35.426378 23826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:35.426446 23826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.426515 23826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:35.426561 23826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.436086 23826 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:40489
I20260807 15:18:35.436107 24025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:40489 every 8 connection(s)
I20260807 15:18:35.437115 23826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:18:35.439687 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 23826
I20260807 15:18:35.440196 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:33331
--local_ip_for_outbound_sockets=127.31.230.195
--tserver_master_addrs=127.31.230.254:44645
--webserver_port=44721
--webserver_interface=127.31.230.195
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:18:35.441509 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.439416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:48399)
0807 15:18:35.440074 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.440213 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":168,"threads_started":1}
W20260807 15:18:35.442109 24026 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:44645 (0 consecutive failures): Network error: Failed to ping master at 127.31.230.254:44645: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
I20260807 15:18:35.442819 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.442521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:43449)
0807 15:18:35.442655 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.442726 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 15:18:35.443517 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.443263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:43383)
0807 15:18:35.443391 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.443451 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:18:35.443692 24026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:18:35.555361 24031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:35.555672 24031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:35.555758 24031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:35.559856 24031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:35.559950 24031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:35.560046 24031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:18:35.565212 24031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:33331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=44721
--tserver_master_addrs=127.31.230.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:35.566313 24031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:35.567436 24031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:35.577204 24039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:35.577229 24043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:35.577291 24038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:35.577378 24031 server_base.cc:1034] running on GCE node
I20260807 15:18:35.577885 24031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:35.578552 24031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:35.579730 24031 hybrid_clock.cc:648] HybridClock initialized: now 1786115915579697 us; error 45 us; skew 500 ppm
Aug 07 15:18:35 dist-test-slave-gkw7 krb5kdc[21985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115915, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:35.582413 24031 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:18:35.583284 24031 webserver.cc:460] Webserver started at http://127.31.230.195:44721/ using document root <none> and password file <none>
I20260807 15:18:35.583847 24031 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:35.583935 24031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:35.587927 24031 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:18:35.589991 24051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.591171 24031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:18:35.591308 24031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e"
format_stamp: "Formatted at 2026-08-07 15:17:53 on dist-test-slave-gkw7"
server_key: "b8e617e6635749d21d850cdb203f2d35"
server_key_iv: "e77a4117547c9f94897511546a8712aa"
server_key_version: "encryptionkey@0"
I20260807 15:18:35.591832 24031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:35.624862 24031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:35.628288 24031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:35.628494 24031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:35.629096 24031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:35.629964 24031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:35.630012 24031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.630052 24031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:35.630074 24031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:35.639266 24031 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:33331
I20260807 15:18:35.639295 24214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:33331 every 8 connection(s)
I20260807 15:18:35.640285 24031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:18:35.644375 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.642593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:39677)
0807 15:18:35.643065 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.643175 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":124,"threads_started":1}
W20260807 15:18:35.644833 24215 heartbeater.cc:644] Failed to heartbeat to 127.31.230.254:44645 (0 consecutive failures): Network error: Failed to ping master at 127.31.230.254:44645: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
I20260807 15:18:35.645432 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.645231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:50743)
0807 15:18:35.645333 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.645380 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:18:35.645999 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:35.645824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:54949)
0807 15:18:35.645914 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:35.645954 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:18:35.646171 24215 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:18:35.648829 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 24031
I20260807 15:18:36.296236 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:36.295720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:56269)
0807 15:18:36.295973 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:36.296085 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260807 15:18:36.444689 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:36.444276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:37673)
0807 15:18:36.444531 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:36.444586 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260807 15:18:36.647125 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:36.646715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:43513)
0807 15:18:36.646965 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:36.647020 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260807 15:18:37.021246 24224 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:37.021483 24224 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:37.022735 24224 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
Aug 07 15:18:37 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115915, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.194@KRBTEST.COM
Aug 07 15:18:37 dist-test-slave-gkw7 krb5kdc[21985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115915, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.195@KRBTEST.COM
I20260807 15:18:37.036870 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.022973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:33331 (local address 127.31.230.193:60795)
0807 15:18:37.023331 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:37.023347 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:37.023471 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:37.025088 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:37.025094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:37.025702 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:37.026200 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:37.026216 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.027499 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.027502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:37.028422 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:37.028432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.029148 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.029152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:37.029318 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:37.030197 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:37.030219 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:37.032224 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:37.034414 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:37.034419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:37.034421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:37.034711 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:37.035079 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:37.035082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:37.035084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:37.035169 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:37.035796 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:37.035806 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:37.036073 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 15:18:37.036280 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 15:18:37.036655 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":246,"spinlock_wait_cycles":4736,"thread_start_us":83,"threads_started":1}
I20260807 15:18:37.036880 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.023164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:60795 (local address 127.31.230.195:33331)
0807 15:18:37.023636 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:37.023657 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:37.023696 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:18:37.024378 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:37.024383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:37.024723 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:37.025031 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:37.026379 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.027353 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.028558 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.029019 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.029194 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:37.032445 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:37.032451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:37.032462 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:37.032492 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:37.034292 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:37.034848 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:37.034859 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:37.034863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:37.034975 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:37.035270 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:37.035273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:37.035275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:37.035666 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:37.035825 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:37.036385 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 15:18:37.036655 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":107,"threads_started":1}
I20260807 15:18:37.036880 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.023185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:32769 (local address 127.31.230.194:40489)
0807 15:18:37.023677 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:37.023687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:37.023721 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:18:37.024383 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:37.024389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:37.024724 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:37.025031 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:37.026376 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.027353 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.028542 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.029000 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.029185 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:37.032178 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:37.032184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:37.032197 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:37.032226 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:37.034149 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:37.034750 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:37.034760 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:37.034764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:37.034880 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:37.035248 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:37.035251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:37.035253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:37.035665 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:37.035821 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:37.036387 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 15:18:37.036670 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":150,"threads_started":1}
I20260807 15:18:37.036921 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.022973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:40489 (local address 127.31.230.193:32769)
0807 15:18:37.023425 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:37.023441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:37.023543 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:37.025086 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:37.025092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:37.025709 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:37.026201 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:37.026217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.027503 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.027507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:37.028418 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:37.028427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:37.029133 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:37.029137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:37.029317 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:37.030203 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:37.030221 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:37.031921 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:37.034283 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:37.034290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:37.034304 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:37.034627 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:37.034981 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:37.034984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:37.034986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:37.035131 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:37.035795 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:37.035802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:37.036074 (+ 272us) client_negotiation.cc:769] Sending connection context
0807 15:18:37.036280 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 15:18:37.036740 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":257,"spinlock_wait_cycles":37632,"thread_start_us":64,"threads_started":1}
W20260807 15:18:37.039817 23666 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
W20260807 15:18:37.039988 23663 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:37.040066 23663 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1; no voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e
I20260807 15:18:37.040328 24224 raft_consensus.cc:2726] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:18:37.297289 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.296972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:35875)
0807 15:18:37.297122 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:37.297188 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 15:18:37.446388 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.445829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:47811)
0807 15:18:37.446175 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:37.446246 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":113,"threads_started":1}
I20260807 15:18:37.648290 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:37.647882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:56841)
0807 15:18:37.648131 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:37.648198 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260807 15:18:38.298532 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:38.298102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:50081)
0807 15:18:38.298372 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:38.298432 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
I20260807 15:18:38.447638 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:38.447171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:54173)
0807 15:18:38.447478 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:38.447541 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260807 15:18:38.649617 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:38.649069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:47455)
0807 15:18:38.649357 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:38.649420 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
I20260807 15:18:38.816632 24238 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:38.816784 24238 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:38.817379 24238 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
W20260807 15:18:38.818320 23666 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
W20260807 15:18:38.818461 23663 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:38.818567 23663 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1; no voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e
I20260807 15:18:38.818760 24238 raft_consensus.cc:2726] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:18:39.299819 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:39.299390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:52215)
0807 15:18:39.299664 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:39.299722 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
I20260807 15:18:39.448987 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:39.448533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:43615)
0807 15:18:39.448810 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:39.448871 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
I20260807 15:18:39.650894 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:39.650434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:51233)
0807 15:18:39.650718 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:39.650783 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":76,"threads_started":1}
I20260807 15:18:40.301365 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:40.300859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:57803)
0807 15:18:40.301192 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:40.301255 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
I20260807 15:18:40.450398 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:40.449927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:53673)
0807 15:18:40.450232 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:40.450296 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260807 15:18:40.652382 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:40.651906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:50969)
0807 15:18:40.652193 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:40.652282 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":98,"threads_started":1}
I20260807 15:18:40.829195 24254 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:40.829353 24254 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:40.829789 24254 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
W20260807 15:18:40.830693 23666 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
W20260807 15:18:40.830804 23663 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:40.830866 23663 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1; no voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e
I20260807 15:18:40.831012 24254 raft_consensus.cc:2726] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:18:41.302587 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:41.302142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:52769)
0807 15:18:41.302432 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:41.302488 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260807 15:18:41.451872 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:41.451342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:59965)
0807 15:18:41.451694 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:41.451765 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260807 15:18:41.653906 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:41.653304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:41999)
0807 15:18:41.653708 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:41.653789 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":102,"threads_started":1}
I20260807 15:18:42.304100 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:42.303583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:45835)
0807 15:18:42.303916 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:42.303987 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":131,"threads_started":1}
I20260807 15:18:42.453342 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:42.452782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:45997)
0807 15:18:42.453174 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:42.453239 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":92,"threads_started":1}
I20260807 15:18:42.655309 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:42.654765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:50163)
0807 15:18:42.655113 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:42.655179 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":96,"threads_started":1}
I20260807 15:18:43.305362 24264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:43.304920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:50647)
0807 15:18:43.305180 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:43.305263 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260807 15:18:43.454716 24265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:43.454245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:36887)
0807 15:18:43.454549 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:43.454612 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":97,"threads_started":1}
I20260807 15:18:43.656827 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:43.656351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:58437)
0807 15:18:43.656655 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:43.656727 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":129,"threads_started":1}
I20260807 15:18:44.306720 24267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:44.306273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:57829)
0807 15:18:44.306547 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:44.306606 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":95,"threads_started":1}
I20260807 15:18:44.456030 24268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:44.455571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:32885)
0807 15:18:44.455868 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:44.455930 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":115,"threads_started":1}
I20260807 15:18:44.658281 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:44.657723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:54409)
0807 15:18:44.658089 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:44.658175 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260807 15:18:45.308048 24272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:45.307589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:39217)
0807 15:18:45.307879 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:45.307944 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
I20260807 15:18:45.457406 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:45.456882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:34361)
0807 15:18:45.457187 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:45.457303 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260807 15:18:45.659570 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:45.659119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:41043)
0807 15:18:45.659412 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:45.659472 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
I20260807 15:18:45.729357 24275 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:45.729501 24275 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:45.729977 24275 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
W20260807 15:18:45.730811 23666 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
W20260807 15:18:45.730932 23663 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:45.731060 23663 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1; no voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e
I20260807 15:18:45.731276 24275 raft_consensus.cc:2726] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:18:46.309435 24276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:46.308989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:55483)
0807 15:18:46.309278 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:46.309337 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260807 15:18:46.458734 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:46.458220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:55225)
0807 15:18:46.458545 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:46.458615 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":86,"threads_started":1}
I20260807 15:18:46.660876 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:46.660416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:48493)
0807 15:18:46.660707 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:46.660771 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
I20260807 15:18:47.310685 24281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:47.310237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:45321)
0807 15:18:47.310524 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:47.310584 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
I20260807 15:18:47.460139 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:47.459605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:38585)
0807 15:18:47.459959 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:47.460029 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":99,"threads_started":1}
I20260807 15:18:47.561297 24287 raft_consensus.cc:482] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:47.561436 24287 raft_consensus.cc:504] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9a9b26381a4f0bb9571da086355eb9" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 40489 } } peers { permanent_uuid: "aa735e5a22ff43538e5f7d9f94a9bd3e" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 33331 } } peers { permanent_uuid: "05b1ece7ca9845b3987a8316b98a99b1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38993 } }
I20260807 15:18:47.561973 24287 leader_election.cc:288] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489), aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331)
W20260807 15:18:47.562810 23663 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aa735e5a22ff43538e5f7d9f94a9bd3e (127.31.230.195:33331): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
W20260807 15:18:47.562950 23666 leader_election.cc:341] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f9a9b26381a4f0bb9571da086355eb9 (127.31.230.194:40489): Not found: Tablet not found: 68db851cc87745aebf3c8924f8e88d33
I20260807 15:18:47.563155 23666 leader_election.cc:302] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b1ece7ca9845b3987a8316b98a99b1; no voters: 3f9a9b26381a4f0bb9571da086355eb9, aa735e5a22ff43538e5f7d9f94a9bd3e
I20260807 15:18:47.563328 24287 raft_consensus.cc:2726] T 68db851cc87745aebf3c8924f8e88d33 P 05b1ece7ca9845b3987a8316b98a99b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:18:47.662341 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:47.661786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:60737)
0807 15:18:47.662157 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:47.662231 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":98,"threads_started":1}
I20260807 15:18:48.312083 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:48.311553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:45169)
0807 15:18:48.311825 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:48.311892 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260807 15:18:48.461406 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:48.460966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:50097)
0807 15:18:48.461239 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:48.461305 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260807 15:18:48.663604 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:48.663117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:58359)
0807 15:18:48.663407 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:48.663477 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":77,"threads_started":1}
I20260807 15:18:49.313369 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:49.312863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:51509)
0807 15:18:49.313196 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:49.313267 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":92,"threads_started":1}
I20260807 15:18:49.462811 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:49.462317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:44469)
0807 15:18:49.462640 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:49.462707 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":104,"threads_started":1}
I20260807 15:18:49.664947 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:49.664451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:45009)
0807 15:18:49.664764 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:49.664831 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":89,"threads_started":1}
I20260807 15:18:50.314883 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:50.314424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.193:45609)
0807 15:18:50.314727 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:50.314786 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":104,"threads_started":1}
I20260807 15:18:50.464331 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:50.463857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.194:48349)
0807 15:18:50.464170 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:50.464231 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260807 15:18:50.650672 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 23582
I20260807 15:18:50.666524 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:50.666061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44645 (local address 127.31.230.195:58671)
0807 15:18:50.666362 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:50.666425 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.254:44645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260807 15:18:50.667577 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 23826
I20260807 15:18:50.680184 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 24031
2026-08-07T15:18:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59616 ms)
[----------] 30 tests from SecurityITest (249934 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:18:50 dist-test-slave-gkw7 krb5kdc[24314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:18:50 dist-test-slave-gkw7 krb5kdc[24314](info): set up 2 sockets
Aug 07 15:18:50 dist-test-slave-gkw7 krb5kdc[24314](info): commencing operation
krb5kdc: starting...
W20260807 15:18:51.972259 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:18:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:18:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:18:52.108458 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44817
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:44817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:52.222566 24331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:52.222828 24331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:52.222867 24331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:52.226962 24331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:18:52.227018 24331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:52.227037 24331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:52.227054 24331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:18:52.227069 24331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:18:52.232311 24331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:44817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:44817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:52.233436 24331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:52.234522 24331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:52.244256 24339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:52.244324 24336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:52.244310 24331 server_base.cc:1034] running on GCE node
W20260807 15:18:52.244256 24337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:52.245199 24331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:52.246408 24331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:52.247606 24331 hybrid_clock.cc:648] HybridClock initialized: now 1786115932247583 us; error 50 us; skew 500 ppm
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:52.250506 24331 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:18:52.251390 24331 webserver.cc:460] Webserver started at http://127.31.230.254:41341/ using document root <none> and password file <none>
I20260807 15:18:52.251977 24331 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:52.252074 24331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:52.252318 24331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:52.254491 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "6dcdf56cdc324b74904b919b46ce6097"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "867293139ee94f695d6af27693e53055"
server_key_iv: "b6128a55ccde583420ca59a6d650d140"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.255159 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "6dcdf56cdc324b74904b919b46ce6097"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "867293139ee94f695d6af27693e53055"
server_key_iv: "b6128a55ccde583420ca59a6d650d140"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.259209 24331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:18:52.261790 24347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.262974 24331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:18:52.263120 24331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "6dcdf56cdc324b74904b919b46ce6097"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "867293139ee94f695d6af27693e53055"
server_key_iv: "b6128a55ccde583420ca59a6d650d140"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.263250 24331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:52.282545 24331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:52.286022 24331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:52.286249 24331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:52.294149 24331 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:44817
I20260807 15:18:52.294225 24421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:44817 every 8 connection(s)
I20260807 15:18:52.295212 24331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:18:52.295928 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 24331
I20260807 15:18:52.296030 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:18:52.296330 32667 external_mini_cluster.cc:1428] Setting key ac58b939b4c365437740d85cb9cf1a7f
I20260807 15:18:52.299074 24422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:52.305989 24422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097: Bootstrap starting.
I20260807 15:18:52.308988 24422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:52.310158 24422 log.cc:824] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:52.311724 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:52.297982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.31.230.254:44817)
0807 15:18:52.298470 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:52.298485 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:52.298521 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:18:52.299157 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:52.299183 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:52.299575 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:52.299919 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:52.300765 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.301868 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.302742 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.303228 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.303405 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:52.306530 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:52.306535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:52.306548 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:52.306577 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:52.308413 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:52.309041 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:52.309045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:52.309050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:52.309234 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:52.309634 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:52.309637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:52.309638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:52.310031 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:52.310202 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:52.310566 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:18:52.310900 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":172,"threads_started":1}
I20260807 15:18:52.312551 24422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097: No bootstrap required, opened a new log
I20260807 15:18:52.316478 24422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcdf56cdc324b74904b919b46ce6097" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44817 } }
I20260807 15:18:52.316756 24422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:52.316845 24422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcdf56cdc324b74904b919b46ce6097, State: Initialized, Role: FOLLOWER
I20260807 15:18:52.317348 24422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcdf56cdc324b74904b919b46ce6097" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44817 } }
I20260807 15:18:52.317510 24422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:18:52.317598 24422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:18:52.317706 24422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:52.318693 24422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcdf56cdc324b74904b919b46ce6097" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44817 } }
I20260807 15:18:52.319060 24422 leader_election.cc:302] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dcdf56cdc324b74904b919b46ce6097; no voters:
I20260807 15:18:52.319412 24422 leader_election.cc:288] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:18:52.319612 24429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:52.319983 24429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [term 1 LEADER]: Becoming Leader. State: Replica: 6dcdf56cdc324b74904b919b46ce6097, State: Running, Role: LEADER
I20260807 15:18:52.320366 24429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcdf56cdc324b74904b919b46ce6097" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44817 } }
I20260807 15:18:52.320721 24422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:18:52.322330 24431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dcdf56cdc324b74904b919b46ce6097. Latest consensus state: current_term: 1 leader_uuid: "6dcdf56cdc324b74904b919b46ce6097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcdf56cdc324b74904b919b46ce6097" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44817 } } }
I20260807 15:18:52.322374 24430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dcdf56cdc324b74904b919b46ce6097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcdf56cdc324b74904b919b46ce6097" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 44817 } } }
I20260807 15:18:52.322487 24431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:52.322487 24430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:52.323709 24446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:18:52.326781 24446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:18:52.332130 24446 catalog_manager.cc:1347] Generated new cluster ID: 7bd607db844145c08c478191cc1c1c6c
I20260807 15:18:52.332207 24446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:18:52.340942 24446 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:18:52.342300 24446 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:18:52.350127 24446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097: Generated new TSK 0
I20260807 15:18:52.351019 24446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:18:52.388777 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44817
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:52.507663 24460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:52.507936 24460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:52.507978 24460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:52.511914 24460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:52.511973 24460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:52.512041 24460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:18:52.517421 24460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:52.518483 24460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:52.519464 24460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:52.530078 24474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:52.530088 24469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:52.530124 24471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:52.530184 24460 server_base.cc:1034] running on GCE node
I20260807 15:18:52.530825 24460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:52.531548 24460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:52.532768 24460 hybrid_clock.cc:648] HybridClock initialized: now 1786115932532730 us; error 51 us; skew 500 ppm
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:52.535720 24460 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:18:52.536639 24460 webserver.cc:460] Webserver started at http://127.31.230.193:40259/ using document root <none> and password file <none>
I20260807 15:18:52.537374 24460 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:52.537462 24460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:52.537774 24460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:52.540059 24460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "d75cbc601aee4953ac7235f0cbc90e5d"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "763e98dce15d9de0afa71e5dc3065023"
server_key_iv: "cad78358a3991c2f1557d1cc32d92e0e"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.540602 24460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "d75cbc601aee4953ac7235f0cbc90e5d"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "763e98dce15d9de0afa71e5dc3065023"
server_key_iv: "cad78358a3991c2f1557d1cc32d92e0e"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.544447 24460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:18:52.546913 24482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.548318 24460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:18:52.548432 24460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "d75cbc601aee4953ac7235f0cbc90e5d"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "763e98dce15d9de0afa71e5dc3065023"
server_key_iv: "cad78358a3991c2f1557d1cc32d92e0e"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.548519 24460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:52.568956 24460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:52.572388 24460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:52.572605 24460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:52.573228 24460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:52.574141 24460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:52.574190 24460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.574230 24460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:52.574283 24460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.583487 24460 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:35265
I20260807 15:18:52.583515 24644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:35265 every 8 connection(s)
I20260807 15:18:52.584429 24460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:18:52.586632 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 24460
I20260807 15:18:52.586776 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:18:52.587100 32667 external_mini_cluster.cc:1428] Setting key 5c14b2f6cb77b7ca858d3477e92c7a09
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:18:52.599494 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:52.586444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:33335 (local address 127.31.230.254:44817)
0807 15:18:52.586591 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:52.586595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:52.587713 (+ 1118us) server_negotiation.cc:407] Connection header received
0807 15:18:52.589156 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:52.589159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:52.589221 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:52.589304 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:52.590516 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.591062 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.592210 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.592452 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.592564 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:52.595472 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:52.595476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:52.595479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:52.595511 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:52.597219 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:52.597846 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:52.597850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:52.597852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:52.597952 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:52.598332 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:52.598335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:52.598336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:52.598541 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:52.598633 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:52.599170 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 15:18:52.599310 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:18:52.600409 24648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:52.586964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44817 (local address 127.31.230.193:33335)
0807 15:18:52.587511 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:52.587573 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:52.588938 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:52.589491 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:52.589498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:52.589885 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:52.590366 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:52.590378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.591191 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.591195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:52.592081 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:52.592090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.592609 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.592613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:52.592787 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:52.593477 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:52.593493 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:52.595295 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:52.597378 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:52.597382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:52.597425 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:52.597747 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:52.598059 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:52.598062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:52.598064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:52.598214 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:52.598643 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:52.598648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:52.598917 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 15:18:52.599108 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 15:18:52.599455 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":152,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:18:52.601903 24645 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44817
I20260807 15:18:52.602159 24645 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:52.602675 24645 heartbeater.cc:511] Master 127.31.230.254:44817 requested a full tablet report, sending...
I20260807 15:18:52.604547 24370 ts_manager.cc:194] Registered new tserver with Master: d75cbc601aee4953ac7235f0cbc90e5d (127.31.230.193:35265)
I20260807 15:18:52.606402 24370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:33335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:18:52.620645 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44817
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:52.744159 24653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:52.744431 24653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:52.744527 24653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:52.748608 24653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:52.748665 24653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:52.748726 24653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:18:52.754185 24653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:52.755223 24653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:52.756258 24653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:52.767063 24661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:52.767222 24653 server_base.cc:1034] running on GCE node
W20260807 15:18:52.767035 24664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:52.767050 24660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:52.767791 24653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:52.768502 24653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:52.769619 24653 hybrid_clock.cc:648] HybridClock initialized: now 1786115932769598 us; error 26 us; skew 500 ppm
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:52.772397 24653 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:18:52.773211 24653 webserver.cc:460] Webserver started at http://127.31.230.194:46523/ using document root <none> and password file <none>
I20260807 15:18:52.773839 24653 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:52.773904 24653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:52.774139 24653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:52.776134 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "bc6ad785033e4b3984fe0edecb9cc4a4"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "c22b4f69cd2d39861a6331d7d22e9e2d"
server_key_iv: "9b1c908f8f036c48d867211390206685"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.776749 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "bc6ad785033e4b3984fe0edecb9cc4a4"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "c22b4f69cd2d39861a6331d7d22e9e2d"
server_key_iv: "9b1c908f8f036c48d867211390206685"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.780515 24653 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 15:18:52.782948 24673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.784190 24653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:18:52.784317 24653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "bc6ad785033e4b3984fe0edecb9cc4a4"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "c22b4f69cd2d39861a6331d7d22e9e2d"
server_key_iv: "9b1c908f8f036c48d867211390206685"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.784428 24653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:52.802024 24653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:52.805119 24653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:52.805344 24653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:52.806073 24653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:52.806967 24653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:52.807018 24653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.807060 24653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:52.807078 24653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:52.817116 24653 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:38601
I20260807 15:18:52.817180 24834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:38601 every 8 connection(s)
I20260807 15:18:52.818138 24653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:18:52.818879 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 24653
I20260807 15:18:52.819041 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:18:52.819412 32667 external_mini_cluster.cc:1428] Setting key e8016543e70713ac30491bfdf804b407
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:18:52.833189 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:52.820364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59123 (local address 127.31.230.254:44817)
0807 15:18:52.820520 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:52.820525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:52.821455 (+ 930us) server_negotiation.cc:407] Connection header received
0807 15:18:52.823027 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:52.823031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:52.823095 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:52.823176 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:52.824361 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.824913 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.826178 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.826424 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.826547 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:52.829137 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:52.829140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:52.829143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:52.829177 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:52.830874 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:52.831461 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:52.831464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:52.831467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:52.831554 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:52.831911 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:52.831915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:52.831916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:52.832215 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:52.832342 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:52.832876 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 15:18:52.833003 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:18:52.834336 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:52.820772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44817 (local address 127.31.230.194:59123)
0807 15:18:52.821313 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:52.821356 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:52.822813 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:52.823355 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:52.823361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:52.823717 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:52.824216 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:52.824228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.825042 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.825046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:52.826048 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:52.826058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:52.826602 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:52.826606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:52.826782 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:52.827424 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:52.827442 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:52.828953 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:52.830998 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:52.831005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:52.831020 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:52.831350 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:52.831649 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:52.831653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:52.831655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:52.831803 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:52.832352 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:52.832361 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:52.832627 (+ 266us) client_negotiation.cc:769] Sending connection context
0807 15:18:52.832831 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 15:18:52.833175 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":154,"threads_started":1}
I20260807 15:18:52.835626 24835 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44817
I20260807 15:18:52.835906 24835 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:52.836465 24835 heartbeater.cc:511] Master 127.31.230.254:44817 requested a full tablet report, sending...
I20260807 15:18:52.837651 24370 ts_manager.cc:194] Registered new tserver with Master: bc6ad785033e4b3984fe0edecb9cc4a4 (127.31.230.194:38601)
I20260807 15:18:52.838272 24370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:59123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:18:52.852596 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44817
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:52.967903 24841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:52.968145 24841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:52.968187 24841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:52.972132 24841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:52.972186 24841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:52.972242 24841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:18:52.977262 24841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:44817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:52.978323 24841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:52.979382 24841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:52.989480 24847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:52.989581 24855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:52.989516 24846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:52.989905 24841 server_base.cc:1034] running on GCE node
I20260807 15:18:52.990335 24841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:52.990964 24841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:52.992134 24841 hybrid_clock.cc:648] HybridClock initialized: now 1786115932992099 us; error 52 us; skew 500 ppm
Aug 07 15:18:52 dist-test-slave-gkw7 krb5kdc[24314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:52.994957 24841 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:18:52.995748 24841 webserver.cc:460] Webserver started at http://127.31.230.195:45477/ using document root <none> and password file <none>
I20260807 15:18:52.996255 24841 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:52.996312 24841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:52.996479 24841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:52.998600 24841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "bfcd598b17484a4881d18c6dd67ba827"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "c40b25fe58b8df710c3993bde1a19a6f"
server_key_iv: "48b642150c35de5f89c6ef14b6a2a111"
server_key_version: "encryptionkey@0"
I20260807 15:18:52.999125 24841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "bfcd598b17484a4881d18c6dd67ba827"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "c40b25fe58b8df710c3993bde1a19a6f"
server_key_iv: "48b642150c35de5f89c6ef14b6a2a111"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.002770 24841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:18:53.005116 24863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:53.006436 24841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:18:53.006583 24841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "bfcd598b17484a4881d18c6dd67ba827"
format_stamp: "Formatted at 2026-08-07 15:18:52 on dist-test-slave-gkw7"
server_key: "c40b25fe58b8df710c3993bde1a19a6f"
server_key_iv: "48b642150c35de5f89c6ef14b6a2a111"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.006697 24841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:53.023085 24841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:53.026152 24841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:53.026360 24841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:53.026985 24841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:53.027829 24841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:53.027902 24841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:53.027976 24841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:53.028023 24841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:53.037197 24841 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:45981
I20260807 15:18:53.037259 25021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:45981 every 8 connection(s)
I20260807 15:18:53.038164 24841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:18:53.039839 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 24841
I20260807 15:18:53.039978 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:18:53.040378 32667 external_mini_cluster.cc:1428] Setting key ee210fd47292f55b2613b997cb8bb045
Aug 07 15:18:53 dist-test-slave-gkw7 krb5kdc[24314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115932, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:53.053014 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.040162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:34435 (local address 127.31.230.254:44817)
0807 15:18:53.040312 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.040317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.041316 (+ 999us) server_negotiation.cc:407] Connection header received
0807 15:18:53.042954 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.042957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.043001 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.043071 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:53.044318 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.044856 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.046197 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.046438 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.046517 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.048996 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:53.048999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:53.049001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:53.049026 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:53.050744 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:53.051353 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:53.051356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:53.051357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:53.051436 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:53.051812 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:53.051814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:53.051816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:53.051985 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:53.052062 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.052709 (+ 647us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.052853 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:18:53.054188 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.040555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:44817 (local address 127.31.230.195:34435)
0807 15:18:53.041133 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.041190 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:53.042756 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:53.043306 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:53.043314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:53.043670 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:53.044189 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.044204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.044973 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.044976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.046070 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.046081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.046599 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.046604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.046750 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.047315 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:53.047331 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:53.048827 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:53.050866 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:53.050871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:53.050883 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:53.051250 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:53.051566 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:53.051569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:53.051570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:53.051724 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:53.052092 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:53.052098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:53.052403 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 15:18:53.052665 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 15:18:53.052981 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":157,"threads_started":1}
I20260807 15:18:53.055306 25022 heartbeater.cc:347] Connected to a master server at 127.31.230.254:44817
I20260807 15:18:53.055531 25022 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:53.055970 25022 heartbeater.cc:511] Master 127.31.230.254:44817 requested a full tablet report, sending...
I20260807 15:18:53.056963 24370 ts_manager.cc:194] Registered new tserver with Master: bfcd598b17484a4881d18c6dd67ba827 (127.31.230.195:45981)
I20260807 15:18:53.057484 24370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:34435
I20260807 15:18:53.066563 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:18:53.076236 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.068626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.31.230.254:44817)
0807 15:18:53.068769 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.068772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.068940 (+ 168us) server_negotiation.cc:407] Connection header received
0807 15:18:53.069103 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.069105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.069143 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.069217 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:53.069952 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.070469 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.071285 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.071491 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.071566 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.073122 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:53.073124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:53.073126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:53.073147 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:53.074672 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:53.075191 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:53.075194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:53.075195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:53.075268 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:53.075509 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:53.075512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:53.075513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:53.075715 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:53.075819 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.076014 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.076106 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:18:53.079036 24370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:18:53.082163 24370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:18:53.096248 25045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.090517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.31.230.194:38601)
0807 15:18:53.091186 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.091200 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.091224 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:18:53.091305 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.091310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.091516 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.091687 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:53.092225 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.093228 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.094721 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.095560 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.095695 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.095831 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.096001 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.096094 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":107,"threads_started":1}
I20260807 15:18:53.096238 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.090411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:38601 (local address 127.0.0.1:34502)
0807 15:18:53.090864 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.090910 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:53.091035 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:53.091911 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:53.091916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:53.091942 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:53.092089 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.092095 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.093357 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.093363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.094580 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.094591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.095691 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.095694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.095799 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.095837 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:18:53.095971 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:18:53.096094 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":68,"threads_started":1}
I20260807 15:18:53.096892 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.090488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:45981 (local address 127.0.0.1:34994)
0807 15:18:53.091437 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.091450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:53.091544 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:53.092390 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:53.092393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:53.092410 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:53.092511 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.092517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.093990 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.093993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.095309 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.095323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.096413 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.096419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.096515 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.096623 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:18:53.096701 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:18:53.096769 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":204,"thread_start_us":106,"threads_started":1}
I20260807 15:18:53.096961 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.091405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.31.230.195:45981)
0807 15:18:53.091870 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.091884 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.091917 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:18:53.092004 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.092009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.092205 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.092394 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:53.092640 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.093840 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.095444 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.096262 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.096596 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.096725 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.096765 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.096829 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":154,"threads_started":1}
I20260807 15:18:53.097286 25043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.091122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:35265 (local address 127.0.0.1:34828)
0807 15:18:53.092120 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.092132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:53.092246 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:53.092657 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:53.092660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:53.092677 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:53.092748 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.092753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.094033 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.094036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.095099 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.095109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.096467 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.096470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.096579 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.097025 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 15:18:53.097095 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:18:53.097163 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":419,"thread_start_us":110,"threads_started":1}
I20260807 15:18:53.097702 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.091365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.31.230.193:35265)
0807 15:18:53.091790 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.091805 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.092252 (+ 447us) server_negotiation.cc:407] Connection header received
0807 15:18:53.092349 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.092354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.092529 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.092690 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:53.092832 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.093921 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.095230 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.096054 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.096177 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.096634 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.097437 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.097562 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":86,"threads_started":1}
I20260807 15:18:53.099542 24748 tablet_service.cc:1467] Processing CreateTablet for tablet 50d70341284542e2853540d3bd639ba2 (DEFAULT_TABLE table=test-table [id=04b4b96c5ce641028c60de91a4f410b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:53.100466 24748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d70341284542e2853540d3bd639ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:53.103374 24935 tablet_service.cc:1467] Processing CreateTablet for tablet 50d70341284542e2853540d3bd639ba2 (DEFAULT_TABLE table=test-table [id=04b4b96c5ce641028c60de91a4f410b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:53.103645 24553 tablet_service.cc:1467] Processing CreateTablet for tablet 50d70341284542e2853540d3bd639ba2 (DEFAULT_TABLE table=test-table [id=04b4b96c5ce641028c60de91a4f410b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:53.104415 24935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d70341284542e2853540d3bd639ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:53.104560 24553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d70341284542e2853540d3bd639ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:53.107931 25049 tablet_bootstrap.cc:492] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Bootstrap starting.
I20260807 15:18:53.110466 25049 tablet_bootstrap.cc:654] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:53.111646 25050 tablet_bootstrap.cc:492] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Bootstrap starting.
I20260807 15:18:53.112418 25049 log.cc:824] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:53.113137 25051 tablet_bootstrap.cc:492] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: Bootstrap starting.
I20260807 15:18:53.113713 25050 tablet_bootstrap.cc:654] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:53.114709 25049 tablet_bootstrap.cc:492] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: No bootstrap required, opened a new log
I20260807 15:18:53.114916 25049 ts_tablet_manager.cc:1397] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:18:53.114926 25051 tablet_bootstrap.cc:654] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:53.114945 25050 log.cc:824] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:53.115720 25051 log.cc:824] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:53.116958 25050 tablet_bootstrap.cc:492] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: No bootstrap required, opened a new log
I20260807 15:18:53.117146 25050 ts_tablet_manager.cc:1397] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260807 15:18:53.117385 25051 tablet_bootstrap.cc:492] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: No bootstrap required, opened a new log
I20260807 15:18:53.117596 25051 ts_tablet_manager.cc:1397] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 15:18:53.118891 25049 raft_consensus.cc:348] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.119179 25049 raft_consensus.cc:374] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:53.119246 25049 raft_consensus.cc:729] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6ad785033e4b3984fe0edecb9cc4a4, State: Initialized, Role: FOLLOWER
I20260807 15:18:53.119699 25049 consensus_queue.cc:260] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.120395 24835 heartbeater.cc:503] Master 127.31.230.254:44817 was elected leader, sending a full tablet report...
I20260807 15:18:53.120805 25049 ts_tablet_manager.cc:1428] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:18:53.121031 25050 raft_consensus.cc:348] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.121310 25050 raft_consensus.cc:374] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:53.121371 25050 raft_consensus.cc:729] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfcd598b17484a4881d18c6dd67ba827, State: Initialized, Role: FOLLOWER
I20260807 15:18:53.121861 25050 consensus_queue.cc:260] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.122555 25022 heartbeater.cc:503] Master 127.31.230.254:44817 was elected leader, sending a full tablet report...
I20260807 15:18:53.122833 25050 ts_tablet_manager.cc:1428] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 15:18:53.123252 25051 raft_consensus.cc:348] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.123514 25051 raft_consensus.cc:374] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:53.123584 25051 raft_consensus.cc:729] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75cbc601aee4953ac7235f0cbc90e5d, State: Initialized, Role: FOLLOWER
I20260807 15:18:53.124517 25051 consensus_queue.cc:260] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.125208 24645 heartbeater.cc:503] Master 127.31.230.254:44817 was elected leader, sending a full tablet report...
I20260807 15:18:53.125567 25051 ts_tablet_manager.cc:1428] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260807 15:18:53.289204 25024 tablet.cc:2369] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:18:53.319377 24836 tablet.cc:2369] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:18:53.335839 24647 tablet.cc:2369] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:18:53.427812 25056 raft_consensus.cc:482] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:53.428066 25056 raft_consensus.cc:504] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.429318 25056 leader_election.cc:288] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfcd598b17484a4881d18c6dd67ba827 (127.31.230.195:45981), d75cbc601aee4953ac7235f0cbc90e5d (127.31.230.193:35265)
I20260807 15:18:53.433756 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.430294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59095 (local address 127.31.230.193:35265)
0807 15:18:53.430425 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.430430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.430573 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:18:53.430764 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.430767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.430810 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.430896 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:53.431179 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.431687 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.432763 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.433168 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.433298 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.433356 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.433561 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.433636 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:18:53.433772 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.430142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:35265 (local address 127.31.230.194:59095)
0807 15:18:53.430453 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.430467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:53.430615 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:53.430908 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:53.430910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:53.430926 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:53.431056 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.431061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.431805 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.431810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.432629 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.432638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.433293 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.433297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.433400 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.433429 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:18:53.433519 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:18:53.433609 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":18,"thread_start_us":81,"threads_started":1}
I20260807 15:18:53.433869 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.429917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:55723 (local address 127.31.230.195:45981)
0807 15:18:53.430083 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.430088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.430183 (+ 95us) server_negotiation.cc:407] Connection header received
0807 15:18:53.430409 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.430413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.430477 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.430609 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:53.430921 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.431493 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.432738 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.433210 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.433349 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.433413 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.433651 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.433731 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:18:53.433941 25059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.429684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:45981 (local address 127.31.230.194:55723)
0807 15:18:53.430000 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.430020 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:53.430275 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:53.430629 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:53.430632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:53.430653 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:53.430797 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.430803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.431620 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.431627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.432597 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:53.432607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.433344 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.433348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:53.433464 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.433497 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 15:18:53.433635 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 15:18:53.433715 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":33,"thread_start_us":119,"threads_started":1}
I20260807 15:18:53.434732 24578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d70341284542e2853540d3bd639ba2" candidate_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" is_pre_election: true
I20260807 15:18:53.434732 24963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d70341284542e2853540d3bd639ba2" candidate_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfcd598b17484a4881d18c6dd67ba827" is_pre_election: true
I20260807 15:18:53.435072 24963 raft_consensus.cc:2454] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6ad785033e4b3984fe0edecb9cc4a4 in term 0.
I20260807 15:18:53.435072 24578 raft_consensus.cc:2454] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6ad785033e4b3984fe0edecb9cc4a4 in term 0.
I20260807 15:18:53.435598 24675 leader_election.cc:302] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc6ad785033e4b3984fe0edecb9cc4a4, d75cbc601aee4953ac7235f0cbc90e5d; no voters:
I20260807 15:18:53.435846 25056 raft_consensus.cc:2781] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:18:53.435933 25056 raft_consensus.cc:482] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:18:53.435966 25056 raft_consensus.cc:3037] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:53.437057 25056 raft_consensus.cc:504] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.437408 25056 leader_election.cc:288] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [CANDIDATE]: Term 1 election: Requested vote from peers bfcd598b17484a4881d18c6dd67ba827 (127.31.230.195:45981), d75cbc601aee4953ac7235f0cbc90e5d (127.31.230.193:35265)
I20260807 15:18:53.437865 24963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d70341284542e2853540d3bd639ba2" candidate_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfcd598b17484a4881d18c6dd67ba827"
I20260807 15:18:53.437978 24578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d70341284542e2853540d3bd639ba2" candidate_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75cbc601aee4953ac7235f0cbc90e5d"
I20260807 15:18:53.438011 24963 raft_consensus.cc:3037] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:53.438105 24578 raft_consensus.cc:3037] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:53.439472 24963 raft_consensus.cc:2454] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6ad785033e4b3984fe0edecb9cc4a4 in term 1.
I20260807 15:18:53.439476 24578 raft_consensus.cc:2454] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6ad785033e4b3984fe0edecb9cc4a4 in term 1.
I20260807 15:18:53.439913 24675 leader_election.cc:302] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc6ad785033e4b3984fe0edecb9cc4a4, d75cbc601aee4953ac7235f0cbc90e5d; no voters:
I20260807 15:18:53.440141 25056 raft_consensus.cc:2781] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:53.440424 25056 raft_consensus.cc:686] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 1 LEADER]: Becoming Leader. State: Replica: bc6ad785033e4b3984fe0edecb9cc4a4, State: Running, Role: LEADER
I20260807 15:18:53.440788 25056 consensus_queue.cc:237] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } }
I20260807 15:18:53.444355 24372 catalog_manager.cc:5495] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc6ad785033e4b3984fe0edecb9cc4a4 (127.31.230.194). New cstate: current_term: 1 leader_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6ad785033e4b3984fe0edecb9cc4a4" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 38601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260807 15:18:53.458714 25045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.454957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34512 (local address 127.31.230.194:38601)
0807 15:18:53.455103 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.455108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.455124 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:18:53.455320 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.455322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.455381 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.455498 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:18:53.455759 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.456273 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.457156 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.457380 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.457491 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.457856 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:18:53.458293 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:18:53.458435 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.458498 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.458557 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:18:53.465060 24578 raft_consensus.cc:1261] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 1 FOLLOWER]: Refusing update from remote peer bc6ad785033e4b3984fe0edecb9cc4a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:53.465111 24963 raft_consensus.cc:1261] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 1 FOLLOWER]: Refusing update from remote peer bc6ad785033e4b3984fe0edecb9cc4a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:53.465847 25056 consensus_queue.cc:1035] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfcd598b17484a4881d18c6dd67ba827" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:53.466034 25062 consensus_queue.cc:1035] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75cbc601aee4953ac7235f0cbc90e5d" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:53.476212 25068 mvcc.cc:205] Tried to move back new op lower bound from 7315930863464919040 to 7315930863376371712. Current Snapshot: MvccSnapshot[applied={T|T < 7315930863464919040}]
I20260807 15:18:53.476529 25069 mvcc.cc:205] Tried to move back new op lower bound from 7315930863464919040 to 7315930863376371712. Current Snapshot: MvccSnapshot[applied={T|T < 7315930863464919040}]
I20260807 15:18:53.483678 24370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:53.483837 24370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:53.486536 24370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dcdf56cdc324b74904b919b46ce6097: Sending DeleteTablet for 3 replicas of tablet 50d70341284542e2853540d3bd639ba2
I20260807 15:18:53.487534 24553 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d70341284542e2853540d3bd639ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34828
I20260807 15:18:53.487560 24748 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d70341284542e2853540d3bd639ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34502
I20260807 15:18:53.487567 24935 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d70341284542e2853540d3bd639ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:53 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:34994
I20260807 15:18:53.488122 25084 tablet_replica.cc:330] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d: stopping tablet replica
I20260807 15:18:53.488400 25083 tablet_replica.cc:330] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: stopping tablet replica
I20260807 15:18:53.488492 25084 raft_consensus.cc:2229] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:53.488658 25083 raft_consensus.cc:2229] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:18:53.488848 25084 raft_consensus.cc:2258] T 50d70341284542e2853540d3bd639ba2 P d75cbc601aee4953ac7235f0cbc90e5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:53.488994 25083 raft_consensus.cc:2258] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:53.489249 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 24460
I20260807 15:18:53.495961 25082 tablet_replica.cc:330] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: stopping tablet replica
I20260807 15:18:53.496443 25082 raft_consensus.cc:2229] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:53.496928 25082 raft_consensus.cc:2258] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:53.498505 25083 ts_tablet_manager.cc:1905] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:53.499117 25082 ts_tablet_manager.cc:1905] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:53.501269 25083 ts_tablet_manager.cc:1918] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:53.501408 25083 log.cc:1196] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/50d70341284542e2853540d3bd639ba2
I20260807 15:18:53.501793 25083 ts_tablet_manager.cc:1939] T 50d70341284542e2853540d3bd639ba2 P bc6ad785033e4b3984fe0edecb9cc4a4: Deleting consensus metadata
I20260807 15:18:53.502862 25082 ts_tablet_manager.cc:1918] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:53.502967 25082 log.cc:1196] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/50d70341284542e2853540d3bd639ba2
I20260807 15:18:53.503137 24355 catalog_manager.cc:4841] TS bc6ad785033e4b3984fe0edecb9cc4a4 (127.31.230.194:38601): tablet 50d70341284542e2853540d3bd639ba2 (table test-table [id=04b4b96c5ce641028c60de91a4f410b1]) successfully deleted
I20260807 15:18:53.503355 25082 ts_tablet_manager.cc:1939] T 50d70341284542e2853540d3bd639ba2 P bfcd598b17484a4881d18c6dd67ba827: Deleting consensus metadata
I20260807 15:18:53.504758 24349 catalog_manager.cc:4841] TS bfcd598b17484a4881d18c6dd67ba827 (127.31.230.195:45981): tablet 50d70341284542e2853540d3bd639ba2 (table test-table [id=04b4b96c5ce641028c60de91a4f410b1]) successfully deleted
W20260807 15:18:53.506134 24349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:35265 (error 108)
I20260807 15:18:53.506301 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 24653
I20260807 15:18:53.511395 25043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.511014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:35265 (local address 127.0.0.1:34840)
0807 15:18:53.511220 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:53.511292 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:35265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 15:18:53.511561 24349 catalog_manager.cc:4584] TS d75cbc601aee4953ac7235f0cbc90e5d (127.31.230.193:35265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50d70341284542e2853540d3bd639ba2: Network error: Client connection negotiation failed: client connection to 127.31.230.193:35265: connect: Connection refused (error 111)
I20260807 15:18:53.517992 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 24841
I20260807 15:18:53.538203 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 24331
2026-08-07T15:18:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2862 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T15:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:18:53Z Disabled control of system clock
I20260807 15:18:53.581082 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33543
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:46559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:33543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:18:53.694532 25091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:53.694799 25091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:53.694834 25091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:53.698839 25091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:18:53.698894 25091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:53.698909 25091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:53.698926 25091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:18:53.698941 25091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:18:53.703706 25091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:33543
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:33543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:53.704733 25091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:53.706113 25091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:53.715018 25106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:53.715049 25101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:53.715018 25104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:53.715196 25091 server_base.cc:1034] running on GCE node
I20260807 15:18:53.716032 25091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:53.717166 25091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:53.718365 25091 hybrid_clock.cc:648] HybridClock initialized: now 1786115933718349 us; error 27 us; skew 500 ppm
I20260807 15:18:53.720357 25091 webserver.cc:460] Webserver started at http://127.31.230.254:43729/ using document root <none> and password file <none>
I20260807 15:18:53.720963 25091 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:53.721060 25091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:53.721307 25091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:53.723474 25091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "f8e5ce6cc9644228b9408d0b0c42b2da"
format_stamp: "Formatted at 2026-08-07 15:18:53 on dist-test-slave-gkw7"
server_key: "afa2c8ee0d34aae15b7c05eee5c3f068"
server_key_iv: "107f6c7321169116c129fcc85a7017cc"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.724064 25091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "f8e5ce6cc9644228b9408d0b0c42b2da"
format_stamp: "Formatted at 2026-08-07 15:18:53 on dist-test-slave-gkw7"
server_key: "afa2c8ee0d34aae15b7c05eee5c3f068"
server_key_iv: "107f6c7321169116c129fcc85a7017cc"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.727828 25091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 15:18:53.730296 25112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:53.731528 25091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:18:53.731698 25091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "f8e5ce6cc9644228b9408d0b0c42b2da"
format_stamp: "Formatted at 2026-08-07 15:18:53 on dist-test-slave-gkw7"
server_key: "afa2c8ee0d34aae15b7c05eee5c3f068"
server_key_iv: "107f6c7321169116c129fcc85a7017cc"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.731814 25091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:53.760180 25091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:53.760979 25091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:53.761200 25091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:53.769399 25091 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:33543
I20260807 15:18:53.769411 25186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:33543 every 8 connection(s)
I20260807 15:18:53.770365 25091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:18:53.773604 25187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:53.777691 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 25091
I20260807 15:18:53.777834 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:18:53.778123 32667 external_mini_cluster.cc:1428] Setting key 8588e2c4271e80cb71562fc4cfe9da42
I20260807 15:18:53.779506 25187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da: Bootstrap starting.
I20260807 15:18:53.781904 25187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:53.782653 25187 log.cc:824] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:53.784526 25187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da: No bootstrap required, opened a new log
I20260807 15:18:53.787652 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:53.779656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.31.230.254:33543)
0807 15:18:53.780295 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:53.780310 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:53.780345 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:18:53.780996 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:53.781012 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:53.781388 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:53.781746 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:53.782207 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.783399 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.784293 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:53.784838 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:53.785036 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:53.785951 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:53.785956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:53.785968 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:53.786351 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:53.786406 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:53.786427 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:53.786536 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:53.786698 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:18:53.786951 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":142,"threads_started":1}
I20260807 15:18:53.788311 25187 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33543 } }
I20260807 15:18:53.788542 25187 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:53.788609 25187 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e5ce6cc9644228b9408d0b0c42b2da, State: Initialized, Role: FOLLOWER
I20260807 15:18:53.789088 25187 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33543 } }
I20260807 15:18:53.789238 25187 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:18:53.789328 25187 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:18:53.789448 25187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:53.790426 25187 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33543 } }
I20260807 15:18:53.790769 25187 leader_election.cc:302] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e5ce6cc9644228b9408d0b0c42b2da; no voters:
I20260807 15:18:53.791102 25187 leader_election.cc:288] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:18:53.791201 25194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:53.791407 25194 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [term 1 LEADER]: Becoming Leader. State: Replica: f8e5ce6cc9644228b9408d0b0c42b2da, State: Running, Role: LEADER
I20260807 15:18:53.791728 25194 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33543 } }
I20260807 15:18:53.792348 25187 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:18:53.792851 25194 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e5ce6cc9644228b9408d0b0c42b2da. Latest consensus state: current_term: 1 leader_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33543 } } }
I20260807 15:18:53.792961 25194 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:53.793236 25197 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e5ce6cc9644228b9408d0b0c42b2da" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 33543 } } }
I20260807 15:18:53.793355 25197 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:53.793340 25202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:18:53.796662 25202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:18:53.802510 25202 catalog_manager.cc:1347] Generated new cluster ID: 2a3cbc026b064e3cbea18984965ec8bc
I20260807 15:18:53.802580 25202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:18:53.807740 25202 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:18:53.808909 25202 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:18:53.814616 25202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da: Generated new TSK 0
I20260807 15:18:53.815527 25202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:18:53.825119 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33543
--builtin_ntp_servers=127.31.230.212:46559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:18:53.941342 25228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:53.941658 25228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:53.941730 25228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:53.945735 25228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:53.945823 25228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:53.945914 25228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:18:53.950682 25228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:53.951838 25228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:53.953295 25228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:53.963706 25236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:53.963821 25228 server_base.cc:1034] running on GCE node
W20260807 15:18:53.963696 25234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:53.963696 25233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:53.964401 25228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:53.965072 25228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:53.966239 25228 hybrid_clock.cc:648] HybridClock initialized: now 1786115933966224 us; error 27 us; skew 500 ppm
I20260807 15:18:53.968092 25228 webserver.cc:460] Webserver started at http://127.31.230.193:42029/ using document root <none> and password file <none>
I20260807 15:18:53.968693 25228 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:53.968758 25228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:53.968977 25228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:53.971124 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "c90e577a70cf49d2914a10043b58f88f"
format_stamp: "Formatted at 2026-08-07 15:18:53 on dist-test-slave-gkw7"
server_key: "fbae45c0b009a9dd37335b6dc75cbd63"
server_key_iv: "b8d5df2b8fa2256f414a7409a71648e7"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.971697 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "c90e577a70cf49d2914a10043b58f88f"
format_stamp: "Formatted at 2026-08-07 15:18:53 on dist-test-slave-gkw7"
server_key: "fbae45c0b009a9dd37335b6dc75cbd63"
server_key_iv: "b8d5df2b8fa2256f414a7409a71648e7"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.975473 25228 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 15:18:53.978065 25242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:53.979414 25228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:18:53.979564 25228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "c90e577a70cf49d2914a10043b58f88f"
format_stamp: "Formatted at 2026-08-07 15:18:53 on dist-test-slave-gkw7"
server_key: "fbae45c0b009a9dd37335b6dc75cbd63"
server_key_iv: "b8d5df2b8fa2256f414a7409a71648e7"
server_key_version: "encryptionkey@0"
I20260807 15:18:53.979694 25228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:53.994382 25228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:53.995110 25228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:53.995324 25228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:53.995957 25228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:53.996933 25228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:53.997005 25228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:53.997080 25228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:53.997124 25228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.006906 25228 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:36277
I20260807 15:18:54.006943 25410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:36277 every 8 connection(s)
I20260807 15:18:54.007838 25228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:18:54.013262 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 25228
I20260807 15:18:54.013399 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:18:54.013754 32667 external_mini_cluster.cc:1428] Setting key d1846fea9a2383f71d197147ed769749
I20260807 15:18:54.015952 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33543
--builtin_ntp_servers=127.31.230.212:46559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:18:54.018622 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.009920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:43655 (local address 127.31.230.254:33543)
0807 15:18:54.010071 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.010074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.011197 (+ 1123us) server_negotiation.cc:407] Connection header received
0807 15:18:54.012679 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.012683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.012747 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.012857 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:54.014193 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.014900 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.016332 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.016726 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.016859 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.018147 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:54.018150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:54.018152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:54.018166 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:54.018186 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:54.018195 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:54.018256 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.018450 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.018505 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:18:54.019817 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.010312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33543 (local address 127.31.230.193:43655)
0807 15:18:54.011025 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.011067 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.012438 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.013056 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.013069 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.013191 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:54.013945 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.013966 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.015050 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.015054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.016177 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.016191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.016894 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.016896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.017052 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.017784 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:18:54.017786 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:54.017857 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:18:54.017957 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:54.018284 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:54.018293 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:54.018301 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.018442 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.018776 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":140,"threads_started":1}
I20260807 15:18:54.021095 25412 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33543
I20260807 15:18:54.021363 25412 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:54.021971 25412 heartbeater.cc:511] Master 127.31.230.254:33543 requested a full tablet report, sending...
I20260807 15:18:54.023742 25130 ts_manager.cc:194] Registered new tserver with Master: c90e577a70cf49d2914a10043b58f88f (127.31.230.193:36277)
I20260807 15:18:54.025568 25130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.193:43655
W20260807 15:18:54.135537 25415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:54.135886 25415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:54.135926 25415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:54.139977 25415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:54.140033 25415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:54.140087 25415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:18:54.144863 25415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:54.145990 25415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:54.147354 25415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:54.157194 25422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:54.157228 25421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:54.157444 25424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:54.157612 25415 server_base.cc:1034] running on GCE node
I20260807 15:18:54.158051 25415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:54.158731 25415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:54.159934 25415 hybrid_clock.cc:648] HybridClock initialized: now 1786115934159914 us; error 62 us; skew 500 ppm
I20260807 15:18:54.161979 25415 webserver.cc:460] Webserver started at http://127.31.230.194:38283/ using document root <none> and password file <none>
I20260807 15:18:54.162606 25415 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:54.162703 25415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:54.162952 25415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:54.165040 25415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "55a4a4af66c84330969a2604c33cc366"
format_stamp: "Formatted at 2026-08-07 15:18:54 on dist-test-slave-gkw7"
server_key: "1fa35fb7b79f5179340ae244dd120835"
server_key_iv: "7932d7a3322fb2ef42288c2ad1dea55a"
server_key_version: "encryptionkey@0"
I20260807 15:18:54.165719 25415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "55a4a4af66c84330969a2604c33cc366"
format_stamp: "Formatted at 2026-08-07 15:18:54 on dist-test-slave-gkw7"
server_key: "1fa35fb7b79f5179340ae244dd120835"
server_key_iv: "7932d7a3322fb2ef42288c2ad1dea55a"
server_key_version: "encryptionkey@0"
I20260807 15:18:54.169464 25415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:18:54.172120 25432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.173400 25415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:18:54.173548 25415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "55a4a4af66c84330969a2604c33cc366"
format_stamp: "Formatted at 2026-08-07 15:18:54 on dist-test-slave-gkw7"
server_key: "1fa35fb7b79f5179340ae244dd120835"
server_key_iv: "7932d7a3322fb2ef42288c2ad1dea55a"
server_key_version: "encryptionkey@0"
I20260807 15:18:54.173729 25415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:54.184155 25415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:54.184804 25415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:54.185026 25415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:54.185696 25415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:54.186635 25415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:54.186705 25415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.186771 25415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:54.186822 25415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.196245 25415 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:46055
I20260807 15:18:54.196269 25587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:46055 every 8 connection(s)
I20260807 15:18:54.197150 25415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:18:54.204452 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 25415
I20260807 15:18:54.204631 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:18:54.204986 32667 external_mini_cluster.cc:1428] Setting key 3589759d9db57b531e20c86ef738221f
I20260807 15:18:54.207072 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.199066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:36677 (local address 127.31.230.254:33543)
0807 15:18:54.199234 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.199238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.200328 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 15:18:54.201876 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.201879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.201928 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.202008 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:54.202874 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.203455 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.204611 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.204955 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.205102 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.206443 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:54.206447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:54.206448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:54.206462 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:54.206526 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:54.206541 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:54.206626 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.206845 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.206914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:18:54.207589 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33543
--builtin_ntp_servers=127.31.230.212:46559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:18:54.207962 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.199510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33543 (local address 127.31.230.194:36677)
0807 15:18:54.200168 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.200217 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.201637 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.202170 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.202179 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.202259 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:54.202744 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.202757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.203567 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.203569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.204433 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.204442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.205144 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.205152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.205310 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.206123 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:18:54.206125 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:54.206177 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:18:54.206257 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:54.206647 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:54.206660 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:54.206669 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.206809 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.207045 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":118,"threads_started":1}
I20260807 15:18:54.209136 25588 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33543
I20260807 15:18:54.209364 25588 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:54.210029 25588 heartbeater.cc:511] Master 127.31.230.254:33543 requested a full tablet report, sending...
I20260807 15:18:54.211112 25130 ts_manager.cc:194] Registered new tserver with Master: 55a4a4af66c84330969a2604c33cc366 (127.31.230.194:46055)
I20260807 15:18:54.211817 25130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.194:36677
W20260807 15:18:54.326297 25592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:54.326579 25592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:54.326648 25592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:54.330647 25592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:54.330724 25592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:54.330806 25592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:18:54.335762 25592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:33543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:54.336853 25592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:54.338336 25592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:54.348186 25602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:54.348337 25592 server_base.cc:1034] running on GCE node
W20260807 15:18:54.348173 25600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:54.348173 25598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:54.348863 25592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:54.349670 25592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:54.350868 25592 hybrid_clock.cc:648] HybridClock initialized: now 1786115934350837 us; error 49 us; skew 500 ppm
I20260807 15:18:54.352896 25592 webserver.cc:460] Webserver started at http://127.31.230.195:35407/ using document root <none> and password file <none>
I20260807 15:18:54.353603 25592 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:54.353700 25592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:54.353940 25592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:54.356175 25592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "a314cf6fd40f4e2eb92868a400b29a28"
format_stamp: "Formatted at 2026-08-07 15:18:54 on dist-test-slave-gkw7"
server_key: "2ed0a2a0f7fb8ef529f0ce2b358dd8cf"
server_key_iv: "66bef8f220d60acb03d85c580a1b46c6"
server_key_version: "encryptionkey@0"
I20260807 15:18:54.356756 25592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "a314cf6fd40f4e2eb92868a400b29a28"
format_stamp: "Formatted at 2026-08-07 15:18:54 on dist-test-slave-gkw7"
server_key: "2ed0a2a0f7fb8ef529f0ce2b358dd8cf"
server_key_iv: "66bef8f220d60acb03d85c580a1b46c6"
server_key_version: "encryptionkey@0"
I20260807 15:18:54.360664 25592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:18:54.363354 25613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.364646 25592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 15:18:54.364785 25592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "a314cf6fd40f4e2eb92868a400b29a28"
format_stamp: "Formatted at 2026-08-07 15:18:54 on dist-test-slave-gkw7"
server_key: "2ed0a2a0f7fb8ef529f0ce2b358dd8cf"
server_key_iv: "66bef8f220d60acb03d85c580a1b46c6"
server_key_version: "encryptionkey@0"
I20260807 15:18:54.364907 25592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:54.380225 25592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:54.380918 25592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:54.381125 25592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:54.381803 25592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:54.382764 25592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:54.382838 25592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.382910 25592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:54.382964 25592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:54.392169 25592 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:41651
I20260807 15:18:54.392215 25769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:41651 every 8 connection(s)
I20260807 15:18:54.393059 25592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:18:54.396183 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 25592
I20260807 15:18:54.396306 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:18:54.396597 32667 external_mini_cluster.cc:1428] Setting key 04fa888addd1a4df03dae4011fa7f2e5
I20260807 15:18:54.402670 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.395103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:45355 (local address 127.31.230.254:33543)
0807 15:18:54.395261 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.395265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.396390 (+ 1125us) server_negotiation.cc:407] Connection header received
0807 15:18:54.398044 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.398047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.398105 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.398174 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:54.399128 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.399673 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.400763 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.400989 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.401067 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.402243 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:54.402246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:54.402248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:54.402260 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:54.402280 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:54.402288 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:54.402350 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.402490 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.402543 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:18:54.403537 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.395545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:33543 (local address 127.31.230.195:45355)
0807 15:18:54.396209 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.396278 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.397825 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.398344 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.398351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.398434 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:54.399001 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.399015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.399785 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.399788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.400659 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.400668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.401117 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.401120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.401342 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.401998 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:18:54.402000 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:54.402040 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:18:54.402101 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:54.402366 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:54.402372 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:54.402376 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.402472 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.402684 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
I20260807 15:18:54.404719 25770 heartbeater.cc:347] Connected to a master server at 127.31.230.254:33543
I20260807 15:18:54.404963 25770 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:54.405470 25770 heartbeater.cc:511] Master 127.31.230.254:33543 requested a full tablet report, sending...
I20260807 15:18:54.406531 25130 ts_manager.cc:194] Registered new tserver with Master: a314cf6fd40f4e2eb92868a400b29a28 (127.31.230.195:41651)
I20260807 15:18:54.407094 25130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.195:45355
I20260807 15:18:54.410720 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:18:54.416729 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.412709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.31.230.254:33543)
0807 15:18:54.412846 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.412853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.413091 (+ 238us) server_negotiation.cc:407] Connection header received
0807 15:18:54.413252 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.413256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.413301 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.413394 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:54.413727 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.414249 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.415105 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.415341 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.415455 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.416301 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:54.416304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:54.416307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:54.416320 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:54.416349 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:54.416358 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:54.416436 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.416526 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.416590 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:18:54.419734 25130 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:18:54.422842 25130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:18:54.437947 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.431546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:46055 (local address 127.0.0.1:35184)
0807 15:18:54.432029 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.432083 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.432227 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.433270 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.433276 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.433301 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:54.433471 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.433478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.434988 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.434993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.436441 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.436453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.437447 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.437450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.437570 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.437593 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.437650 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.437771 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":54,"thread_start_us":140,"threads_started":1}
I20260807 15:18:54.437994 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.431908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36277 (local address 127.0.0.1:53398)
0807 15:18:54.432326 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.432340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.432432 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.433564 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.433567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.433583 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:54.433695 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.433701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.435023 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.435026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.436276 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.436286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.437617 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.437621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.437713 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.437734 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.437804 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.437867 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":181,"thread_start_us":90,"threads_started":1}
I20260807 15:18:54.438272 25785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.431757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35184 (local address 127.31.230.194:46055)
0807 15:18:54.432702 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.432717 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.432745 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:18:54.432822 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.432828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.433049 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.433286 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:54.433621 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.434842 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.436573 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.437304 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.437757 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.437912 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.437963 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.438083 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":128,"threads_started":1}
I20260807 15:18:54.438634 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.432417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.31.230.193:36277)
0807 15:18:54.432896 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.432909 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.432931 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:18:54.433011 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.433017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.433213 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.433378 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:54.433819 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.434905 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.436433 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.437304 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.437439 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.437651 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.438364 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.438474 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":150,"threads_started":1}
I20260807 15:18:54.438635 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.432803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.31.230.195:41651)
0807 15:18:54.433193 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.433212 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.433570 (+ 358us) server_negotiation.cc:407] Connection header received
0807 15:18:54.433649 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.433655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.433836 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.433991 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:54.434214 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.435047 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.436610 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.437159 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.437309 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.437503 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.437858 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.437963 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":160,"threads_started":1}
I20260807 15:18:54.439147 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.432339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41651 (local address 127.0.0.1:51190)
0807 15:18:54.433347 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.433367 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.433451 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.433971 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.433974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.433992 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:54.434081 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.434089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.435183 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.435186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.436441 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.436453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.437296 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.437300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.437417 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.437454 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.437604 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.437765 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260807 15:18:54.441434 25679 tablet_service.cc:1467] Processing CreateTablet for tablet 5d69c861d69641dca60ff99a64244e3f (DEFAULT_TABLE table=test-table [id=cb2ff13c7865456da23b56d92bf1966e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:54.441454 25498 tablet_service.cc:1467] Processing CreateTablet for tablet 5d69c861d69641dca60ff99a64244e3f (DEFAULT_TABLE table=test-table [id=cb2ff13c7865456da23b56d92bf1966e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:54.441447 25311 tablet_service.cc:1467] Processing CreateTablet for tablet 5d69c861d69641dca60ff99a64244e3f (DEFAULT_TABLE table=test-table [id=cb2ff13c7865456da23b56d92bf1966e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:54.442615 25498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d69c861d69641dca60ff99a64244e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:54.442615 25311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d69c861d69641dca60ff99a64244e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:54.442690 25679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d69c861d69641dca60ff99a64244e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:54.449172 25790 tablet_bootstrap.cc:492] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Bootstrap starting.
I20260807 15:18:54.449896 25791 tablet_bootstrap.cc:492] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Bootstrap starting.
I20260807 15:18:54.450564 25792 tablet_bootstrap.cc:492] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: Bootstrap starting.
I20260807 15:18:54.452253 25792 tablet_bootstrap.cc:654] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:54.452258 25790 tablet_bootstrap.cc:654] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:54.452258 25791 tablet_bootstrap.cc:654] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:54.453174 25791 log.cc:824] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:54.453166 25790 log.cc:824] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:54.453171 25792 log.cc:824] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:54.455053 25791 tablet_bootstrap.cc:492] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: No bootstrap required, opened a new log
I20260807 15:18:54.455054 25792 tablet_bootstrap.cc:492] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: No bootstrap required, opened a new log
I20260807 15:18:54.455268 25792 ts_tablet_manager.cc:1397] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 15:18:54.455273 25791 ts_tablet_manager.cc:1397] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.005s
I20260807 15:18:54.456501 25790 tablet_bootstrap.cc:492] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: No bootstrap required, opened a new log
I20260807 15:18:54.456689 25790 ts_tablet_manager.cc:1397] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260807 15:18:54.458716 25792 raft_consensus.cc:348] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.458972 25792 raft_consensus.cc:374] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:54.459026 25792 raft_consensus.cc:729] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90e577a70cf49d2914a10043b58f88f, State: Initialized, Role: FOLLOWER
I20260807 15:18:54.459533 25792 consensus_queue.cc:260] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.459647 25791 raft_consensus.cc:348] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.459964 25791 raft_consensus.cc:374] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:54.460016 25791 raft_consensus.cc:729] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a4a4af66c84330969a2604c33cc366, State: Initialized, Role: FOLLOWER
I20260807 15:18:54.460485 25791 consensus_queue.cc:260] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.460690 25792 ts_tablet_manager.cc:1428] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 15:18:54.461063 25412 heartbeater.cc:503] Master 127.31.230.254:33543 was elected leader, sending a full tablet report...
I20260807 15:18:54.460919 25790 raft_consensus.cc:348] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.461174 25790 raft_consensus.cc:374] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:54.461239 25790 raft_consensus.cc:729] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a314cf6fd40f4e2eb92868a400b29a28, State: Initialized, Role: FOLLOWER
I20260807 15:18:54.461460 25791 ts_tablet_manager.cc:1428] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 15:18:54.461604 25588 heartbeater.cc:503] Master 127.31.230.254:33543 was elected leader, sending a full tablet report...
I20260807 15:18:54.462620 25790 consensus_queue.cc:260] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.463290 25770 heartbeater.cc:503] Master 127.31.230.254:33543 was elected leader, sending a full tablet report...
I20260807 15:18:54.463608 25790 ts_tablet_manager.cc:1428] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
W20260807 15:18:54.509028 25413 tablet.cc:2369] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:18:54.529995 25796 raft_consensus.cc:482] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:54.530256 25796 raft_consensus.cc:504] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.531524 25796 leader_election.cc:288] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55a4a4af66c84330969a2604c33cc366 (127.31.230.194:46055), a314cf6fd40f4e2eb92868a400b29a28 (127.31.230.195:41651)
I20260807 15:18:54.536746 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.532270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:39175 (local address 127.31.230.195:41651)
0807 15:18:54.532415 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.532419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.532495 (+ 76us) server_negotiation.cc:407] Connection header received
0807 15:18:54.532598 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.532601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.532638 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.532728 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:54.532990 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.533724 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.535398 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.536015 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.536157 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.536198 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.536534 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.536607 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:18:54.536759 25785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.531873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:50529 (local address 127.31.230.194:46055)
0807 15:18:54.532006 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.532010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.532204 (+ 194us) server_negotiation.cc:407] Connection header received
0807 15:18:54.532272 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.532276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.532335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.532428 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:54.532757 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.533586 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.535398 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.536026 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.536140 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.536193 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.536534 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.536607 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:18:54.536791 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.531739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:46055 (local address 127.31.230.193:50529)
0807 15:18:54.531989 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.532008 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.532119 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.532466 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.532469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.532496 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:54.532628 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.532634 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.533699 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.533702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.535263 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.535275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.536147 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.536150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.536361 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.536418 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.536495 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.536609 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":253,"thread_start_us":91,"threads_started":1}
I20260807 15:18:54.536791 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.532120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:41651 (local address 127.31.230.193:39175)
0807 15:18:54.532388 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.532423 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:54.532509 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:54.532763 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:54.532765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:54.532782 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:54.532881 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.532885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.533824 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.533826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.535263 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:54.535275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.536147 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.536150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:54.536355 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.536413 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:18:54.536495 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:18:54.536656 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":150,"thread_start_us":88,"threads_started":1}
I20260807 15:18:54.537665 25526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d69c861d69641dca60ff99a64244e3f" candidate_uuid: "c90e577a70cf49d2914a10043b58f88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55a4a4af66c84330969a2604c33cc366" is_pre_election: true
I20260807 15:18:54.537909 25707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d69c861d69641dca60ff99a64244e3f" candidate_uuid: "c90e577a70cf49d2914a10043b58f88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a314cf6fd40f4e2eb92868a400b29a28" is_pre_election: true
I20260807 15:18:54.537998 25526 raft_consensus.cc:2454] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90e577a70cf49d2914a10043b58f88f in term 0.
I20260807 15:18:54.538251 25707 raft_consensus.cc:2454] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90e577a70cf49d2914a10043b58f88f in term 0.
I20260807 15:18:54.538543 25246 leader_election.cc:302] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55a4a4af66c84330969a2604c33cc366, c90e577a70cf49d2914a10043b58f88f; no voters:
I20260807 15:18:54.538795 25796 raft_consensus.cc:2781] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:18:54.538898 25796 raft_consensus.cc:482] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:18:54.538960 25796 raft_consensus.cc:3037] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:54.539999 25796 raft_consensus.cc:504] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.540357 25796 leader_election.cc:288] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [CANDIDATE]: Term 1 election: Requested vote from peers 55a4a4af66c84330969a2604c33cc366 (127.31.230.194:46055), a314cf6fd40f4e2eb92868a400b29a28 (127.31.230.195:41651)
I20260807 15:18:54.540720 25526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d69c861d69641dca60ff99a64244e3f" candidate_uuid: "c90e577a70cf49d2914a10043b58f88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55a4a4af66c84330969a2604c33cc366"
I20260807 15:18:54.540753 25707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d69c861d69641dca60ff99a64244e3f" candidate_uuid: "c90e577a70cf49d2914a10043b58f88f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a314cf6fd40f4e2eb92868a400b29a28"
I20260807 15:18:54.540872 25707 raft_consensus.cc:3037] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:54.540866 25526 raft_consensus.cc:3037] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:54.542019 25707 raft_consensus.cc:2454] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90e577a70cf49d2914a10043b58f88f in term 1.
I20260807 15:18:54.542043 25526 raft_consensus.cc:2454] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90e577a70cf49d2914a10043b58f88f in term 1.
I20260807 15:18:54.542404 25246 leader_election.cc:302] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55a4a4af66c84330969a2604c33cc366, c90e577a70cf49d2914a10043b58f88f; no voters:
I20260807 15:18:54.542573 25796 raft_consensus.cc:2781] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:54.542830 25796 raft_consensus.cc:686] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [term 1 LEADER]: Becoming Leader. State: Replica: c90e577a70cf49d2914a10043b58f88f, State: Running, Role: LEADER
I20260807 15:18:54.543169 25796 consensus_queue.cc:237] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } }
I20260807 15:18:54.546609 25128 catalog_manager.cc:5495] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f reported cstate change: term changed from 0 to 1, leader changed from <none> to c90e577a70cf49d2914a10043b58f88f (127.31.230.193). New cstate: current_term: 1 leader_uuid: "c90e577a70cf49d2914a10043b58f88f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c90e577a70cf49d2914a10043b58f88f" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 36277 } health_report { overall_health: HEALTHY } } }
I20260807 15:18:54.562348 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.557960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.31.230.193:36277)
0807 15:18:54.558109 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:54.558115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:54.558275 (+ 160us) server_negotiation.cc:407] Connection header received
0807 15:18:54.558526 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:54.558529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:54.558603 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:54.558733 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:18:54.559007 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.559590 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.560565 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:54.560824 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:54.560956 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:54.561329 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:18:54.561892 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:18:54.562051 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:54.562122 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:18:54.562185 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:18:54.568897 25526 raft_consensus.cc:1261] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 1 FOLLOWER]: Refusing update from remote peer c90e577a70cf49d2914a10043b58f88f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:54.568925 25707 raft_consensus.cc:1261] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 1 FOLLOWER]: Refusing update from remote peer c90e577a70cf49d2914a10043b58f88f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:54.569635 25796 consensus_queue.cc:1035] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [LEADER]: Connected to new peer: Peer: permanent_uuid: "55a4a4af66c84330969a2604c33cc366" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:54.569835 25802 consensus_queue.cc:1035] T 5d69c861d69641dca60ff99a64244e3f P c90e577a70cf49d2914a10043b58f88f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a314cf6fd40f4e2eb92868a400b29a28" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 41651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:54.576421 25805 mvcc.cc:205] Tried to move back new op lower bound from 7315930867986542592 to 7315930867891494912. Current Snapshot: MvccSnapshot[applied={T|T < 7315930867986542592}]
I20260807 15:18:54.578001 25808 mvcc.cc:205] Tried to move back new op lower bound from 7315930867986542592 to 7315930867891494912. Current Snapshot: MvccSnapshot[applied={T|T < 7315930867986542592}]
I20260807 15:18:54.580262 25806 mvcc.cc:205] Tried to move back new op lower bound from 7315930867986542592 to 7315930867891494912. Current Snapshot: MvccSnapshot[applied={T|T < 7315930867986542592}]
I20260807 15:18:54.586026 25130 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:54.586217 25130 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:54.588822 25130 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8e5ce6cc9644228b9408d0b0c42b2da: Sending DeleteTablet for 3 replicas of tablet 5d69c861d69641dca60ff99a64244e3f
I20260807 15:18:54.589639 25311 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d69c861d69641dca60ff99a64244e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:54 UTC) from {username='slave'} at 127.0.0.1:53398
I20260807 15:18:54.589872 25498 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d69c861d69641dca60ff99a64244e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:54 UTC) from {username='slave'} at 127.0.0.1:35184
I20260807 15:18:54.590153 25679 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d69c861d69641dca60ff99a64244e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:54 UTC) from {username='slave'} at 127.0.0.1:51190
I20260807 15:18:54.590284 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 25228
I20260807 15:18:54.593164 25827 tablet_replica.cc:330] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: stopping tablet replica
I20260807 15:18:54.593520 25827 raft_consensus.cc:2229] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:54.593884 25827 raft_consensus.cc:2258] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:54.594288 25828 tablet_replica.cc:330] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: stopping tablet replica
I20260807 15:18:54.594578 25828 raft_consensus.cc:2229] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:54.594885 25828 raft_consensus.cc:2258] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:54.595618 25827 ts_tablet_manager.cc:1905] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:54.597936 25827 ts_tablet_manager.cc:1918] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:54.598019 25827 log.cc:1196] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/5d69c861d69641dca60ff99a64244e3f
I20260807 15:18:54.598315 25827 ts_tablet_manager.cc:1939] T 5d69c861d69641dca60ff99a64244e3f P 55a4a4af66c84330969a2604c33cc366: Deleting consensus metadata
I20260807 15:18:54.599450 25117 catalog_manager.cc:4841] TS 55a4a4af66c84330969a2604c33cc366 (127.31.230.194:46055): tablet 5d69c861d69641dca60ff99a64244e3f (table test-table [id=cb2ff13c7865456da23b56d92bf1966e]) successfully deleted
W20260807 15:18:54.600406 25116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:36277 (error 108)
I20260807 15:18:54.600626 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 25415
I20260807 15:18:54.608078 25828 ts_tablet_manager.cc:1905] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:54.610452 25828 ts_tablet_manager.cc:1918] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:54.610592 25828 log.cc:1196] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/5d69c861d69641dca60ff99a64244e3f
I20260807 15:18:54.610939 25828 ts_tablet_manager.cc:1939] T 5d69c861d69641dca60ff99a64244e3f P a314cf6fd40f4e2eb92868a400b29a28: Deleting consensus metadata
I20260807 15:18:54.611557 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:54.611211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:36277 (local address 127.0.0.1:53418)
0807 15:18:54.611357 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:54.611449 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 15:18:54.611752 25116 catalog_manager.cc:4584] TS c90e577a70cf49d2914a10043b58f88f (127.31.230.193:36277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d69c861d69641dca60ff99a64244e3f: Network error: Client connection negotiation failed: client connection to 127.31.230.193:36277: connect: Connection refused (error 111)
I20260807 15:18:54.612625 25117 catalog_manager.cc:4841] TS a314cf6fd40f4e2eb92868a400b29a28 (127.31.230.195:41651): tablet 5d69c861d69641dca60ff99a64244e3f (table test-table [id=cb2ff13c7865456da23b56d92bf1966e]) successfully deleted
I20260807 15:18:54.615063 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 25592
I20260807 15:18:54.626187 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 25091
2026-08-07T15:18:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1083 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:18:54 dist-test-slave-gkw7 krb5kdc[25833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:18:54 dist-test-slave-gkw7 krb5kdc[25833](info): set up 2 sockets
Aug 07 15:18:54 dist-test-slave-gkw7 krb5kdc[25833](info): commencing operation
krb5kdc: starting...
W20260807 15:18:55.877717 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:18:55 dist-test-slave-gkw7 krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:18:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:18:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:18:56.016639 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:40111
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:40111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:56.135468 25850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:56.135790 25850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:56.135874 25850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:56.139981 25850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:18:56.140082 25850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:56.140122 25850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:56.140192 25850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:18:56.140228 25850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:18:56.145754 25850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:40111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:40111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:56.146916 25850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:56.148008 25850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:56.157182 25858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.157243 25863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:56.157430 25850 server_base.cc:1034] running on GCE node
W20260807 15:18:56.157204 25859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:56.158155 25850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:56.159375 25850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:56.160571 25850 hybrid_clock.cc:648] HybridClock initialized: now 1786115936160544 us; error 43 us; skew 500 ppm
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:56.167877 25850 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:18:56.168938 25850 webserver.cc:460] Webserver started at http://127.31.230.254:37829/ using document root <none> and password file <none>
I20260807 15:18:56.169631 25850 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:56.169740 25850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:56.170003 25850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:56.172217 25850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "5aaa86f3f1284c2c810920499a036256"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "be5c1f7b3ceee24e13718d8d5c363f6a"
server_key_iv: "da9a2dc04b0bcbf9eb65b817e452f331"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.172833 25850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "5aaa86f3f1284c2c810920499a036256"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "be5c1f7b3ceee24e13718d8d5c363f6a"
server_key_iv: "da9a2dc04b0bcbf9eb65b817e452f331"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.177049 25850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:18:56.179795 25875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.181222 25850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:18:56.181380 25850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "5aaa86f3f1284c2c810920499a036256"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "be5c1f7b3ceee24e13718d8d5c363f6a"
server_key_iv: "da9a2dc04b0bcbf9eb65b817e452f331"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.181509 25850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:56.193084 25850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:56.196594 25850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:56.196843 25850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:56.204859 25850 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:40111
I20260807 15:18:56.204888 25940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:40111 every 8 connection(s)
I20260807 15:18:56.205910 25850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:18:56.208958 25942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:56.214756 25942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256: Bootstrap starting.
I20260807 15:18:56.214932 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 25850
I20260807 15:18:56.215101 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:18:56.215392 32667 external_mini_cluster.cc:1428] Setting key 9476355116c4c864395ba7a7761c1540
I20260807 15:18:56.217500 25942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:56.218298 25942 log.cc:824] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:56.220522 25942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256: No bootstrap required, opened a new log
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:56.225411 25942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaa86f3f1284c2c810920499a036256" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40111 } }
I20260807 15:18:56.225725 25942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:56.225814 25942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaa86f3f1284c2c810920499a036256, State: Initialized, Role: FOLLOWER
I20260807 15:18:56.226336 25942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaa86f3f1284c2c810920499a036256" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40111 } }
I20260807 15:18:56.226519 25942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:18:56.226631 25942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:18:56.226769 25942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:56.227810 25942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaa86f3f1284c2c810920499a036256" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40111 } }
I20260807 15:18:56.228173 25942 leader_election.cc:302] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5aaa86f3f1284c2c810920499a036256; no voters:
I20260807 15:18:56.228504 25942 leader_election.cc:288] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:18:56.228615 25950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:56.228881 25950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [term 1 LEADER]: Becoming Leader. State: Replica: 5aaa86f3f1284c2c810920499a036256, State: Running, Role: LEADER
I20260807 15:18:56.229249 25950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaa86f3f1284c2c810920499a036256" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40111 } }
I20260807 15:18:56.229974 25942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:18:56.231025 25951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5aaa86f3f1284c2c810920499a036256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaa86f3f1284c2c810920499a036256" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40111 } } }
I20260807 15:18:56.231169 25951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:56.231585 25952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5aaa86f3f1284c2c810920499a036256. Latest consensus state: current_term: 1 leader_uuid: "5aaa86f3f1284c2c810920499a036256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaa86f3f1284c2c810920499a036256" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 40111 } } }
I20260807 15:18:56.231745 25952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256 [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:56.231822 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.216824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.31.230.254:40111)
0807 15:18:56.217289 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:56.217305 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:56.217341 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:18:56.218115 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:56.218137 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:56.218524 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:56.218898 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:56.219791 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.220898 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.222155 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.222667 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.222792 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.225953 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:56.225959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:56.225974 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:56.226003 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:56.227866 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.228423 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.228431 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.228439 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.228593 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.228933 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.228937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.228939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.229455 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:56.229703 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:56.230357 (+ 654us) server_negotiation.cc:299] Negotiation successful
0807 15:18:56.230687 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":142,"threads_started":1}
I20260807 15:18:56.232566 25961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:18:56.235761 25961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:18:56.241279 25961 catalog_manager.cc:1347] Generated new cluster ID: 247a7cf600444056b4755673f8c9412d
I20260807 15:18:56.241358 25961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:18:56.253072 25961 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:18:56.254261 25961 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:18:56.270642 25961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256: Generated new TSK 0
I20260807 15:18:56.271473 25961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:18:56.304306 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40111
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:56.422765 25978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:56.423053 25978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:56.423098 25978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:56.427179 25978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:56.427253 25978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:56.427312 25978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:18:56.433189 25978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:56.434412 25978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:56.435432 25978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:56.445511 25989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.445538 25985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.445511 25986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:56.445952 25978 server_base.cc:1034] running on GCE node
I20260807 15:18:56.446381 25978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:56.446996 25978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:56.448173 25978 hybrid_clock.cc:648] HybridClock initialized: now 1786115936448146 us; error 28 us; skew 500 ppm
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:56.451196 25978 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:18:56.452021 25978 webserver.cc:460] Webserver started at http://127.31.230.193:43541/ using document root <none> and password file <none>
I20260807 15:18:56.452605 25978 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:56.452669 25978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:56.452840 25978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:56.454983 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "1409cb9c197643318f70deaf7629397b"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "8c2874d98cde3e9cc3d8201388ec5c61"
server_key_iv: "13b0f709576bb71f85a983d0148ba2ad"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.455530 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "1409cb9c197643318f70deaf7629397b"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "8c2874d98cde3e9cc3d8201388ec5c61"
server_key_iv: "13b0f709576bb71f85a983d0148ba2ad"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.459306 25978 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:18:56.461923 25998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.463508 25978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:18:56.463614 25978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "1409cb9c197643318f70deaf7629397b"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "8c2874d98cde3e9cc3d8201388ec5c61"
server_key_iv: "13b0f709576bb71f85a983d0148ba2ad"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.463774 25978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:56.495294 25978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:56.498904 25978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:56.499145 25978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:56.499995 25978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:56.500888 25978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:56.500943 25978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.501015 25978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:56.501048 25978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.510831 25978 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:37975
I20260807 15:18:56.510854 26142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:37975 every 8 connection(s)
I20260807 15:18:56.511786 25978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:18:56.512253 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 25978
I20260807 15:18:56.512385 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:18:56.512730 32667 external_mini_cluster.cc:1428] Setting key a6025ef3a6f414b6e9f20a39a2c6764b
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:18:56.527112 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.513880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:37987 (local address 127.31.230.254:40111)
0807 15:18:56.514029 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:56.514034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:56.515177 (+ 1143us) server_negotiation.cc:407] Connection header received
0807 15:18:56.516508 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:56.516511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:56.516564 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:56.516645 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:56.517916 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.518463 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.519636 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.519897 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.520006 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.523114 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:56.523118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:56.523120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:56.523149 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:56.524837 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.525413 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.525417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.525419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.525510 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.525920 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.525925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.525929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.526135 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:56.526249 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:56.526777 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 15:18:56.526927 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
I20260807 15:18:56.527987 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.514448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40111 (local address 127.31.230.193:37987)
0807 15:18:56.514957 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:56.515013 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:56.516287 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:56.516820 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:56.516827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:56.517195 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:56.517765 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:56.517781 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.518597 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.518606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:56.519499 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:56.519508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.520064 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.520068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:56.520327 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.521006 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:56.521035 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:56.522919 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:56.524959 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:56.524964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:56.524978 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:56.525288 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:56.525661 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:56.525665 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:56.525667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:56.525816 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:56.526248 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:56.526255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:56.526526 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 15:18:56.526731 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 15:18:56.527082 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":149,"threads_started":1}
I20260807 15:18:56.529443 26143 heartbeater.cc:347] Connected to a master server at 127.31.230.254:40111
I20260807 15:18:56.529739 26143 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:56.530328 26143 heartbeater.cc:511] Master 127.31.230.254:40111 requested a full tablet report, sending...
I20260807 15:18:56.532224 25897 ts_manager.cc:194] Registered new tserver with Master: 1409cb9c197643318f70deaf7629397b (127.31.230.193:37975)
I20260807 15:18:56.534240 25897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:37987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:18:56.546120 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40111
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:56.666143 26149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:56.666482 26149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:56.666538 26149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:56.671247 26149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:56.671324 26149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:56.671389 26149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:18:56.676893 26149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:56.678093 26149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:56.679141 26149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:56.689863 26162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.689850 26159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.689850 26158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:56.690192 26149 server_base.cc:1034] running on GCE node
I20260807 15:18:56.690686 26149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:56.691403 26149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:56.692579 26149 hybrid_clock.cc:648] HybridClock initialized: now 1786115936692560 us; error 37 us; skew 500 ppm
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:56.695704 26149 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:18:56.696717 26149 webserver.cc:460] Webserver started at http://127.31.230.194:40923/ using document root <none> and password file <none>
I20260807 15:18:56.697345 26149 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:56.697408 26149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:56.697719 26149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:56.699908 26149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "eaf3df25cfab44f9973597514b3420a5"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "c42214895d3464ea105cd1efd8dae478"
server_key_iv: "b948f5c1f5a8347c2ce1beedace833f9"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.700511 26149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "eaf3df25cfab44f9973597514b3420a5"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "c42214895d3464ea105cd1efd8dae478"
server_key_iv: "b948f5c1f5a8347c2ce1beedace833f9"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.704530 26149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:18:56.707186 26172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.708511 26149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:18:56.708653 26149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "eaf3df25cfab44f9973597514b3420a5"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "c42214895d3464ea105cd1efd8dae478"
server_key_iv: "b948f5c1f5a8347c2ce1beedace833f9"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.708773 26149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:56.722048 26149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:56.725431 26149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:56.725677 26149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:56.726348 26149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:56.727242 26149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:56.727325 26149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.727421 26149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:56.727474 26149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.737949 26149 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:41093
I20260807 15:18:56.737975 26320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:41093 every 8 connection(s)
I20260807 15:18:56.738935 26149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:18:56.744443 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 26149
I20260807 15:18:56.744626 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:18:56.744954 32667 external_mini_cluster.cc:1428] Setting key ee083ea3771e4ec03a76fbc5f2f0ce52
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:56.755091 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.740865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:37845 (local address 127.31.230.254:40111)
0807 15:18:56.741014 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:56.741018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:56.741977 (+ 959us) server_negotiation.cc:407] Connection header received
0807 15:18:56.743325 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:56.743328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:56.743387 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:56.743477 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:56.744764 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.745389 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.746847 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.747065 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.747152 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.750380 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:56.750383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:56.750385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:56.750410 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:56.751999 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.752605 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.752609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.752611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.752702 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.753236 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.753241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.753244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.753596 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:56.753729 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:56.754842 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0807 15:18:56.754957 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:18:56.756446 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.741276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40111 (local address 127.31.230.194:37845)
0807 15:18:56.741816 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:56.741862 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:56.743102 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:56.743679 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:56.743686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:56.744049 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:56.744566 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:56.744580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.745601 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.745604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:56.746711 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:56.746725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.747243 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.747250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:56.747504 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.748193 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:56.748222 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:56.750193 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:56.752170 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:56.752177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:56.752195 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:56.752481 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:56.752849 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:56.752857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:56.752862 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:56.753101 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:56.753756 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:56.753767 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:56.754472 (+ 705us) client_negotiation.cc:769] Sending connection context
0807 15:18:56.754768 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 15:18:56.755193 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
I20260807 15:18:56.757824 26321 heartbeater.cc:347] Connected to a master server at 127.31.230.254:40111
I20260807 15:18:56.758169 26321 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:56.758782 26321 heartbeater.cc:511] Master 127.31.230.254:40111 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
I20260807 15:18:56.760010 25897 ts_manager.cc:194] Registered new tserver with Master: eaf3df25cfab44f9973597514b3420a5 (127.31.230.194:41093)
I20260807 15:18:56.760689 25897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:37845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:18:56.780521 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40111
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:18:56.902797 26327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:56.903071 26327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:56.903123 26327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:56.907220 26327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:56.907292 26327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:56.907352 26327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:18:56.913120 26327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:56.914338 26327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:56.915460 26327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:56.925509 26334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.925735 26336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:56.925523 26338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:56.925778 26327 server_base.cc:1034] running on GCE node
I20260807 15:18:56.926271 26327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:56.926944 26327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:56.928145 26327 hybrid_clock.cc:648] HybridClock initialized: now 1786115936928128 us; error 30 us; skew 500 ppm
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:18:56.931421 26327 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:18:56.932435 26327 webserver.cc:460] Webserver started at http://127.31.230.195:34339/ using document root <none> and password file <none>
I20260807 15:18:56.932999 26327 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:56.933060 26327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:56.933277 26327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:56.935479 26327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "b7cb5197ce774da0bf9e0023c27cd06c"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "c67be5bd1258aad17abeef97c8ffc358"
server_key_iv: "c22f2784139f64cb26cd2689694eab35"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.936017 26327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "b7cb5197ce774da0bf9e0023c27cd06c"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "c67be5bd1258aad17abeef97c8ffc358"
server_key_iv: "c22f2784139f64cb26cd2689694eab35"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.940225 26327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:18:56.943290 26345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.944839 26327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:18:56.944995 26327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "b7cb5197ce774da0bf9e0023c27cd06c"
format_stamp: "Formatted at 2026-08-07 15:18:56 on dist-test-slave-gkw7"
server_key: "c67be5bd1258aad17abeef97c8ffc358"
server_key_iv: "c22f2784139f64cb26cd2689694eab35"
server_key_version: "encryptionkey@0"
I20260807 15:18:56.945127 26327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:56.958690 26327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:56.962335 26327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:56.962565 26327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:56.963250 26327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:56.964164 26327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:56.964215 26327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.964257 26327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:56.964318 26327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:56.973968 26327 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:36501
I20260807 15:18:56.974004 26502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:36501 every 8 connection(s)
I20260807 15:18:56.974969 26327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:18:56.979027 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 26327
I20260807 15:18:56.979182 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:18:56.979487 32667 external_mini_cluster.cc:1428] Setting key ec51cf97387280fb5094c5bde2d5e972
Aug 07 15:18:56 dist-test-slave-gkw7 krb5kdc[25833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115936, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:18:56.989355 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.976979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:54271 (local address 127.31.230.254:40111)
0807 15:18:56.977136 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:56.977140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:56.978155 (+ 1015us) server_negotiation.cc:407] Connection header received
0807 15:18:56.979957 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:56.979961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:56.980023 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:56.980114 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:56.981446 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.982075 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.983255 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.983500 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.983616 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.986234 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:56.986238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:56.986240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:56.986265 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:56.987541 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.988014 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.988016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.988017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.988089 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:56.988388 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:56.988390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:56.988391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:56.988566 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:56.988643 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:56.989105 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 15:18:56.989226 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:18:56.990329 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:56.977397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:40111 (local address 127.31.230.195:54271)
0807 15:18:56.977957 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:56.978003 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:56.979698 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:56.980284 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:56.980291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:56.980794 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:56.981299 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:56.981313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.982216 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.982224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:56.983128 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:56.983137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:56.983665 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:56.983669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:56.983839 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:56.984435 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:18:56.984451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:56.986046 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:56.987654 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:56.987659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:56.987672 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:56.987928 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:56.988168 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:18:56.988170 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:18:56.988171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:18:56.988306 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:18:56.988655 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:56.988660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:56.988896 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 15:18:56.989057 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 15:18:56.989390 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":195,"threads_started":1}
I20260807 15:18:56.991680 26505 heartbeater.cc:347] Connected to a master server at 127.31.230.254:40111
I20260807 15:18:56.991962 26505 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:56.992858 26505 heartbeater.cc:511] Master 127.31.230.254:40111 requested a full tablet report, sending...
I20260807 15:18:56.994205 25897 ts_manager.cc:194] Registered new tserver with Master: b7cb5197ce774da0bf9e0023c27cd06c (127.31.230.195:36501)
I20260807 15:18:56.994881 25897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:54271
I20260807 15:18:57.006148 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:18:57.017195 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.008449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.31.230.254:40111)
0807 15:18:57.008596 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.008600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.008837 (+ 237us) server_negotiation.cc:407] Connection header received
0807 15:18:57.008888 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.008891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.008940 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.009024 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:57.009801 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.010532 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.011406 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.011648 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.011738 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.013653 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:57.013656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:57.013659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:18:57.013684 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:57.015220 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:57.015732 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:57.015735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:57.015736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:57.015820 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:18:57.016150 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:18:57.016153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:18:57.016154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:18:57.016375 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:57.016504 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.016759 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.016890 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:18:57.020620 25897 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:18:57.024003 25897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:18:57.040814 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.034472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.31.230.193:37975)
0807 15:18:57.034889 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.034897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.035022 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:18:57.035196 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.035202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.035418 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.035577 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:57.036493 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.037572 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.039266 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.039840 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.039981 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.040128 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.040533 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.040650 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":82,"threads_started":1}
I20260807 15:18:57.040798 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.033775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.31.230.195:36501)
0807 15:18:57.035874 (+ 2099us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.035882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.035899 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:18:57.035963 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.035967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.036153 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.036312 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:57.036583 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.037704 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.039014 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.039944 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.040083 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.040247 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.040482 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.040604 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":139,"threads_started":1}
I20260807 15:18:57.040845 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.033725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:37975 (local address 127.0.0.1:35250)
0807 15:18:57.034906 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.034923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:57.035040 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:57.035598 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:57.035608 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:57.035648 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:57.035987 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.035997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.037714 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.037717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.039117 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.039124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.039970 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.039974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.040094 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.040135 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 15:18:57.040260 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 15:18:57.040700 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"mutex_wait_us":196,"spinlock_wait_cycles":8192,"thread_start_us":129,"threads_started":1}
I20260807 15:18:57.041595 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.033627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:36501 (local address 127.0.0.1:39100)
0807 15:18:57.034465 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.034520 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:57.034649 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:57.036319 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:57.036322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:57.036347 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:57.036426 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.036431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.037878 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.037882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.038873 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.038884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.040212 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.040218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.040328 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.040348 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:18:57.040418 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:18:57.041436 (+ 1018us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":906,"thread_start_us":90,"threads_started":1}
I20260807 15:18:57.042016 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.034473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54546 (local address 127.31.230.194:41093)
0807 15:18:57.035163 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.035177 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.035207 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:18:57.035272 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.035276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.035418 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.035577 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:57.036170 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.037381 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.039259 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.040171 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.040310 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.040433 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.041745 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.041870 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":118,"threads_started":1}
I20260807 15:18:57.042181 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.034043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41093 (local address 127.0.0.1:54546)
0807 15:18:57.035022 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.035035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:57.035135 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:57.035598 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:57.035606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:57.035647 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:57.035987 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.035997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.037553 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.037557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.039110 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.039124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.040506 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.040510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.041575 (+ 1065us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.041603 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:18:57.041961 (+ 358us) client_negotiation.cc:240] Negotiation successful
0807 15:18:57.042047 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":1102,"spinlock_wait_cycles":6144,"thread_start_us":94,"threads_started":1}
I20260807 15:18:57.044451 26059 tablet_service.cc:1467] Processing CreateTablet for tablet fd262fd8dd28453b9fad6a97bf0a6224 (DEFAULT_TABLE table=test-table [id=74bba2c78af548d28fb850a2086dadf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:57.044720 26236 tablet_service.cc:1467] Processing CreateTablet for tablet fd262fd8dd28453b9fad6a97bf0a6224 (DEFAULT_TABLE table=test-table [id=74bba2c78af548d28fb850a2086dadf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:57.044708 26413 tablet_service.cc:1467] Processing CreateTablet for tablet fd262fd8dd28453b9fad6a97bf0a6224 (DEFAULT_TABLE table=test-table [id=74bba2c78af548d28fb850a2086dadf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:57.045797 26059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd262fd8dd28453b9fad6a97bf0a6224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:57.045801 26413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd262fd8dd28453b9fad6a97bf0a6224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:57.045863 26236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd262fd8dd28453b9fad6a97bf0a6224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:57.052927 26529 tablet_bootstrap.cc:492] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: Bootstrap starting.
I20260807 15:18:57.054311 26530 tablet_bootstrap.cc:492] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Bootstrap starting.
I20260807 15:18:57.055485 26529 tablet_bootstrap.cc:654] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:57.056149 26530 tablet_bootstrap.cc:654] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:57.056515 26529 log.cc:824] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:57.057297 26530 log.cc:824] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:57.057406 26531 tablet_bootstrap.cc:492] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Bootstrap starting.
I20260807 15:18:57.059190 26530 tablet_bootstrap.cc:492] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: No bootstrap required, opened a new log
I20260807 15:18:57.059345 26531 tablet_bootstrap.cc:654] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:57.059420 26530 ts_tablet_manager.cc:1397] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:18:57.060456 26531 log.cc:824] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:57.062301 26529 tablet_bootstrap.cc:492] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: No bootstrap required, opened a new log
I20260807 15:18:57.062542 26529 ts_tablet_manager.cc:1397] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260807 15:18:57.062628 26531 tablet_bootstrap.cc:492] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: No bootstrap required, opened a new log
I20260807 15:18:57.062842 26531 ts_tablet_manager.cc:1397] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:18:57.062983 26530 raft_consensus.cc:348] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.063261 26530 raft_consensus.cc:374] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:57.063315 26530 raft_consensus.cc:729] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7cb5197ce774da0bf9e0023c27cd06c, State: Initialized, Role: FOLLOWER
I20260807 15:18:57.063789 26530 consensus_queue.cc:260] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.064875 26530 ts_tablet_manager.cc:1428] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:18:57.065305 26505 heartbeater.cc:503] Master 127.31.230.254:40111 was elected leader, sending a full tablet report...
I20260807 15:18:57.066740 26531 raft_consensus.cc:348] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.067060 26531 raft_consensus.cc:374] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:57.067121 26531 raft_consensus.cc:729] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf3df25cfab44f9973597514b3420a5, State: Initialized, Role: FOLLOWER
I20260807 15:18:57.067533 26529 raft_consensus.cc:348] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.067656 26531 consensus_queue.cc:260] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.067819 26529 raft_consensus.cc:374] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:57.067888 26529 raft_consensus.cc:729] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1409cb9c197643318f70deaf7629397b, State: Initialized, Role: FOLLOWER
I20260807 15:18:57.068413 26529 consensus_queue.cc:260] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.068816 26531 ts_tablet_manager.cc:1428] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 15:18:57.069203 26321 heartbeater.cc:503] Master 127.31.230.254:40111 was elected leader, sending a full tablet report...
I20260807 15:18:57.069520 26529 ts_tablet_manager.cc:1428] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:18:57.069902 26143 heartbeater.cc:503] Master 127.31.230.254:40111 was elected leader, sending a full tablet report...
I20260807 15:18:57.167920 26539 raft_consensus.cc:482] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:57.168174 26539 raft_consensus.cc:504] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.169498 26539 leader_election.cc:288] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1409cb9c197643318f70deaf7629397b (127.31.230.193:37975), eaf3df25cfab44f9973597514b3420a5 (127.31.230.194:41093)
I20260807 15:18:57.173789 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.170057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:36033 (local address 127.31.230.194:41093)
0807 15:18:57.170193 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.170196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.170518 (+ 322us) server_negotiation.cc:407] Connection header received
0807 15:18:57.170696 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.170698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.170743 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.170833 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:57.171114 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.171614 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.172698 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.173102 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.173241 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.173294 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.173564 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.173638 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:18:57.173820 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.169915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:54219 (local address 127.31.230.193:37975)
0807 15:18:57.170046 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.170051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.170069 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:18:57.170255 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.170259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.170322 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.170448 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:57.170773 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.171340 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.172611 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.173076 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.173220 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.173279 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.173518 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.173678 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:18:57.173820 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.169745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:37975 (local address 127.31.230.195:54219)
0807 15:18:57.169916 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.169935 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:57.170069 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:57.170459 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:57.170463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:57.170482 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:57.170628 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.170640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.171473 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.171477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.172451 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.172461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.173221 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.173225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.173368 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.173400 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 15:18:57.173498 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:18:57.173678 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":53}
I20260807 15:18:57.173803 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.169960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:41093 (local address 127.31.230.195:36033)
0807 15:18:57.170413 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.170424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:57.170566 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:57.170846 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:57.170848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:57.170866 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:57.170996 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.171001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.171740 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.171743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.172561 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.172570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.173237 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.173241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.173380 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.173418 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:18:57.173511 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:18:57.173615 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":43,"thread_start_us":240,"threads_started":1}
I20260807 15:18:57.174609 26262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd262fd8dd28453b9fad6a97bf0a6224" candidate_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3df25cfab44f9973597514b3420a5" is_pre_election: true
I20260807 15:18:57.174757 26084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd262fd8dd28453b9fad6a97bf0a6224" candidate_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1409cb9c197643318f70deaf7629397b" is_pre_election: true
I20260807 15:18:57.174973 26262 raft_consensus.cc:2454] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7cb5197ce774da0bf9e0023c27cd06c in term 0.
I20260807 15:18:57.175117 26084 raft_consensus.cc:2454] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7cb5197ce774da0bf9e0023c27cd06c in term 0.
I20260807 15:18:57.175509 26348 leader_election.cc:302] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7cb5197ce774da0bf9e0023c27cd06c, eaf3df25cfab44f9973597514b3420a5; no voters:
I20260807 15:18:57.175840 26539 raft_consensus.cc:2781] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:18:57.175927 26539 raft_consensus.cc:482] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:18:57.175961 26539 raft_consensus.cc:3037] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:57.177098 26539 raft_consensus.cc:504] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.177497 26539 leader_election.cc:288] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [CANDIDATE]: Term 1 election: Requested vote from peers 1409cb9c197643318f70deaf7629397b (127.31.230.193:37975), eaf3df25cfab44f9973597514b3420a5 (127.31.230.194:41093)
I20260807 15:18:57.177924 26084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd262fd8dd28453b9fad6a97bf0a6224" candidate_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1409cb9c197643318f70deaf7629397b"
I20260807 15:18:57.177973 26262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd262fd8dd28453b9fad6a97bf0a6224" candidate_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3df25cfab44f9973597514b3420a5"
I20260807 15:18:57.178068 26084 raft_consensus.cc:3037] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:57.178083 26262 raft_consensus.cc:3037] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:57.179220 26084 raft_consensus.cc:2454] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7cb5197ce774da0bf9e0023c27cd06c in term 1.
I20260807 15:18:57.179232 26262 raft_consensus.cc:2454] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7cb5197ce774da0bf9e0023c27cd06c in term 1.
I20260807 15:18:57.179610 26350 leader_election.cc:302] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1409cb9c197643318f70deaf7629397b, b7cb5197ce774da0bf9e0023c27cd06c; no voters:
I20260807 15:18:57.179865 26539 raft_consensus.cc:2781] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:57.180064 26539 raft_consensus.cc:686] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 1 LEADER]: Becoming Leader. State: Replica: b7cb5197ce774da0bf9e0023c27cd06c, State: Running, Role: LEADER
I20260807 15:18:57.180413 26539 consensus_queue.cc:237] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } }
I20260807 15:18:57.184020 25897 catalog_manager.cc:5495] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c reported cstate change: term changed from 0 to 1, leader changed from <none> to b7cb5197ce774da0bf9e0023c27cd06c (127.31.230.195). New cstate: current_term: 1 leader_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7cb5197ce774da0bf9e0023c27cd06c" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 36501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260807 15:18:57.196794 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.192750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39108 (local address 127.31.230.195:36501)
0807 15:18:57.192906 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.192910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.192924 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:18:57.193033 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.193035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.193105 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.193206 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:18:57.193447 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.194181 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.195190 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.195418 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.195537 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.195882 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:18:57.196301 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:18:57.196468 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.196559 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.196636 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:18:57.203451 26262 raft_consensus.cc:1261] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 1 FOLLOWER]: Refusing update from remote peer b7cb5197ce774da0bf9e0023c27cd06c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:57.203450 26084 raft_consensus.cc:1261] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 1 FOLLOWER]: Refusing update from remote peer b7cb5197ce774da0bf9e0023c27cd06c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:57.204273 26539 consensus_queue.cc:1035] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf3df25cfab44f9973597514b3420a5" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:57.204490 26543 consensus_queue.cc:1035] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1409cb9c197643318f70deaf7629397b" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 37975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:57.211462 26546 mvcc.cc:205] Tried to move back new op lower bound from 7315930878777643008 to 7315930878693875712. Current Snapshot: MvccSnapshot[applied={T|T < 7315930878777643008}]
I20260807 15:18:57.216704 26548 mvcc.cc:205] Tried to move back new op lower bound from 7315930878777643008 to 7315930878693875712. Current Snapshot: MvccSnapshot[applied={T|T < 7315930878777643008}]
I20260807 15:18:57.225975 25896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:57.226212 25896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:57.229208 25896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5aaa86f3f1284c2c810920499a036256: Sending DeleteTablet for 3 replicas of tablet fd262fd8dd28453b9fad6a97bf0a6224
I20260807 15:18:57.230194 26413 tablet_service.cc:1514] Processing DeleteTablet for tablet fd262fd8dd28453b9fad6a97bf0a6224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:57 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:39100
I20260807 15:18:57.230266 26059 tablet_service.cc:1514] Processing DeleteTablet for tablet fd262fd8dd28453b9fad6a97bf0a6224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:57 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:35250
I20260807 15:18:57.230336 26236 tablet_service.cc:1514] Processing DeleteTablet for tablet fd262fd8dd28453b9fad6a97bf0a6224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:57 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:54546
I20260807 15:18:57.230938 26559 tablet_replica.cc:330] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b: stopping tablet replica
I20260807 15:18:57.231420 26559 raft_consensus.cc:2229] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:57.231746 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 25978
I20260807 15:18:57.231832 26559 raft_consensus.cc:2258] T fd262fd8dd28453b9fad6a97bf0a6224 P 1409cb9c197643318f70deaf7629397b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:57.241495 26561 tablet_replica.cc:330] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: stopping tablet replica
I20260807 15:18:57.242096 26561 raft_consensus.cc:2229] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:18:57.242789 26561 raft_consensus.cc:2258] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:57.243307 26558 tablet_replica.cc:330] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: stopping tablet replica
I20260807 15:18:57.243798 26558 raft_consensus.cc:2229] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:57.244282 26558 raft_consensus.cc:2258] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:57.259352 26561 ts_tablet_manager.cc:1905] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:57.260481 26558 ts_tablet_manager.cc:1905] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:57.262764 26561 ts_tablet_manager.cc:1918] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:57.262984 26561 log.cc:1196] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/fd262fd8dd28453b9fad6a97bf0a6224
I20260807 15:18:57.263455 26561 ts_tablet_manager.cc:1939] T fd262fd8dd28453b9fad6a97bf0a6224 P b7cb5197ce774da0bf9e0023c27cd06c: Deleting consensus metadata
I20260807 15:18:57.263546 26558 ts_tablet_manager.cc:1918] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:57.263651 26558 log.cc:1196] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/fd262fd8dd28453b9fad6a97bf0a6224
I20260807 15:18:57.263960 26558 ts_tablet_manager.cc:1939] T fd262fd8dd28453b9fad6a97bf0a6224 P eaf3df25cfab44f9973597514b3420a5: Deleting consensus metadata
I20260807 15:18:57.265239 25878 catalog_manager.cc:4841] TS b7cb5197ce774da0bf9e0023c27cd06c (127.31.230.195:36501): tablet fd262fd8dd28453b9fad6a97bf0a6224 (table test-table [id=74bba2c78af548d28fb850a2086dadf1]) successfully deleted
I20260807 15:18:57.265239 25877 catalog_manager.cc:4841] TS eaf3df25cfab44f9973597514b3420a5 (127.31.230.194:41093): tablet fd262fd8dd28453b9fad6a97bf0a6224 (table test-table [id=74bba2c78af548d28fb850a2086dadf1]) successfully deleted
I20260807 15:18:57.275286 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 26149
W20260807 15:18:57.280155 25879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:37975 (error 108)
I20260807 15:18:57.289083 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 26327
I20260807 15:18:57.290050 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.289483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:37975 (local address 127.0.0.1:35266)
0807 15:18:57.289855 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.289940 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270}
W20260807 15:18:57.291515 25879 catalog_manager.cc:4584] TS 1409cb9c197643318f70deaf7629397b (127.31.230.193:37975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd262fd8dd28453b9fad6a97bf0a6224: Network error: Client connection negotiation failed: client connection to 127.31.230.193:37975: connect: Connection refused (error 111)
I20260807 15:18:57.302829 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 25850
2026-08-07T15:18:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2679 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T15:18:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:18:57Z Disabled control of system clock
I20260807 15:18:57.344542 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37197
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:44193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:18:57.462278 26569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:57.462586 26569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:57.462625 26569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:57.466637 26569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:18:57.466708 26569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:57.466724 26569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:57.466742 26569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:18:57.466758 26569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:18:57.471752 26569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:44193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:57.472957 26569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:57.474411 26569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:57.483560 26575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:57.483623 26569 server_base.cc:1034] running on GCE node
W20260807 15:18:57.483568 26581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:57.483548 26578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:57.484452 26569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:57.485718 26569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:57.486928 26569 hybrid_clock.cc:648] HybridClock initialized: now 1786115937486909 us; error 32 us; skew 500 ppm
I20260807 15:18:57.488932 26569 webserver.cc:460] Webserver started at http://127.31.230.254:34483/ using document root <none> and password file <none>
I20260807 15:18:57.489617 26569 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:57.489717 26569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:57.489953 26569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:57.492159 26569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "0635836939c944b8be8579877d6ed71c"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "af765668764445773120326935f26b74"
server_key_iv: "c8106106c1abb25b71c0fdeb4ab1cf5f"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.492797 26569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "0635836939c944b8be8579877d6ed71c"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "af765668764445773120326935f26b74"
server_key_iv: "c8106106c1abb25b71c0fdeb4ab1cf5f"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.496886 26569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:18:57.499727 26591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:57.501186 26569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:18:57.501349 26569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "0635836939c944b8be8579877d6ed71c"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "af765668764445773120326935f26b74"
server_key_iv: "c8106106c1abb25b71c0fdeb4ab1cf5f"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.501482 26569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:57.514832 26569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:57.515632 26569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:57.515883 26569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:57.523836 26657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:37197 every 8 connection(s)
I20260807 15:18:57.523845 26569 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:37197
I20260807 15:18:57.524945 26569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:18:57.528276 26659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:57.531344 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 26569
I20260807 15:18:57.531466 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:18:57.531802 32667 external_mini_cluster.cc:1428] Setting key 855c7c425c6e6f5d1b0a18431fd8415e
I20260807 15:18:57.534960 26659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c: Bootstrap starting.
I20260807 15:18:57.537590 26659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:57.538565 26659 log.cc:824] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:57.541145 26659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c: No bootstrap required, opened a new log
I20260807 15:18:57.542717 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.533574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.31.230.254:37197)
0807 15:18:57.534258 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.534276 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.534316 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:18:57.535035 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.535053 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.535537 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.535865 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:57.536240 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.537290 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.538630 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.539218 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.539359 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.540289 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:57.540296 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:57.540319 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:57.540886 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:57.540966 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:57.540995 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:57.541108 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.541341 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.541700 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":140,"threads_started":1}
I20260807 15:18:57.545989 26659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0635836939c944b8be8579877d6ed71c" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:18:57.546232 26659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:57.546270 26659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0635836939c944b8be8579877d6ed71c, State: Initialized, Role: FOLLOWER
I20260807 15:18:57.546718 26659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0635836939c944b8be8579877d6ed71c" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:18:57.546847 26659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:18:57.546897 26659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:18:57.546972 26659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:57.548156 26659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0635836939c944b8be8579877d6ed71c" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:18:57.548556 26659 leader_election.cc:302] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0635836939c944b8be8579877d6ed71c; no voters:
I20260807 15:18:57.548849 26659 leader_election.cc:288] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:18:57.548985 26665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:57.549283 26665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [term 1 LEADER]: Becoming Leader. State: Replica: 0635836939c944b8be8579877d6ed71c, State: Running, Role: LEADER
I20260807 15:18:57.549659 26665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0635836939c944b8be8579877d6ed71c" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } }
I20260807 15:18:57.550287 26659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:18:57.551474 26669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0635836939c944b8be8579877d6ed71c. Latest consensus state: current_term: 1 leader_uuid: "0635836939c944b8be8579877d6ed71c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0635836939c944b8be8579877d6ed71c" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } } }
I20260807 15:18:57.551643 26669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:57.551975 26667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0635836939c944b8be8579877d6ed71c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0635836939c944b8be8579877d6ed71c" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37197 } } }
I20260807 15:18:57.552075 26667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c [sys.catalog]: This master's current role is: LEADER
I20260807 15:18:57.552284 26683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:18:57.555656 26683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:18:57.561434 26683 catalog_manager.cc:1347] Generated new cluster ID: 0f09ba7e1fc8406aa6e78cbd0ce49f30
I20260807 15:18:57.561532 26683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:18:57.581485 26683 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:18:57.582510 26683 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:18:57.588899 26683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c: Generated new TSK 0
I20260807 15:18:57.589753 26683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:18:57.600538 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37197
--builtin_ntp_servers=127.31.230.212:44193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:18:57.726461 26695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:57.726730 26695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:57.726770 26695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:57.730778 26695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:57.730834 26695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:57.730892 26695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:18:57.735494 26695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:44193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:57.736536 26695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:57.737990 26695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:57.748174 26703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:57.748155 26705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:57.748155 26702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:57.748472 26695 server_base.cc:1034] running on GCE node
I20260807 15:18:57.748946 26695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:57.749677 26695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:57.750907 26695 hybrid_clock.cc:648] HybridClock initialized: now 1786115937750839 us; error 71 us; skew 500 ppm
I20260807 15:18:57.753216 26695 webserver.cc:460] Webserver started at http://127.31.230.193:39045/ using document root <none> and password file <none>
I20260807 15:18:57.753952 26695 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:57.754026 26695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:57.754277 26695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:57.756649 26695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "2022749872aa41a5b0e007adaf23dfe1"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "ae03bce6603aaed288845f8bb3275f89"
server_key_iv: "5de067ea3d0bd50c923b68fcd7887143"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.757285 26695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "2022749872aa41a5b0e007adaf23dfe1"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "ae03bce6603aaed288845f8bb3275f89"
server_key_iv: "5de067ea3d0bd50c923b68fcd7887143"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.761590 26695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:18:57.764195 26713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:57.765522 26695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:18:57.765748 26695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "2022749872aa41a5b0e007adaf23dfe1"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "ae03bce6603aaed288845f8bb3275f89"
server_key_iv: "5de067ea3d0bd50c923b68fcd7887143"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.765874 26695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:57.787037 26695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:57.787822 26695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:57.788040 26695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:57.788684 26695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:57.789759 26695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:57.789832 26695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:57.789927 26695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:57.789976 26695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:57.799273 26695 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:39287
I20260807 15:18:57.799299 26874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:39287 every 8 connection(s)
I20260807 15:18:57.800266 26695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:18:57.809301 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 26695
I20260807 15:18:57.809449 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:18:57.809850 32667 external_mini_cluster.cc:1428] Setting key 842996cc4a1084f8a2ae75a1990d75a3
I20260807 15:18:57.810747 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.802602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:42787 (local address 127.31.230.254:37197)
0807 15:18:57.802769 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:57.802773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:57.803966 (+ 1193us) server_negotiation.cc:407] Connection header received
0807 15:18:57.805484 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:57.805486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:57.805586 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:57.805667 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:57.806720 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.807304 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.808537 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.808783 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.808903 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.810212 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:57.810216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:57.810219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:57.810233 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:57.810261 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:57.810271 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:57.810352 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:57.810537 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 15:18:57.810606 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:18:57.811774 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:57.803035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37197 (local address 127.31.230.193:42787)
0807 15:18:57.803827 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:57.803871 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:57.805232 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:57.805842 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:57.805849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:57.805936 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:57.806577 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.806592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.807433 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.807436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.808395 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:57.808405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:57.808953 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:57.808957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:57.809136 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:57.809869 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:18:57.809871 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:57.809943 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:18:57.810026 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:57.810368 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:57.810380 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:57.810386 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:18:57.810532 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 15:18:57.810856 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":111,"threads_started":1}
I20260807 15:18:57.811959 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37197
--builtin_ntp_servers=127.31.230.212:44193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:18:57.813120 26875 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37197
I20260807 15:18:57.813381 26875 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:57.814073 26875 heartbeater.cc:511] Master 127.31.230.254:37197 requested a full tablet report, sending...
I20260807 15:18:57.815748 26611 ts_manager.cc:194] Registered new tserver with Master: 2022749872aa41a5b0e007adaf23dfe1 (127.31.230.193:39287)
I20260807 15:18:57.817049 26611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.193:42787
W20260807 15:18:57.933348 26880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:57.933811 26880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:57.933902 26880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:57.938167 26880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:57.938234 26880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:57.938294 26880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:18:57.943192 26880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:44193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:57.944540 26880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:57.946053 26880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:57.956481 26890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:57.956509 26888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:57.956481 26887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:57.956758 26880 server_base.cc:1034] running on GCE node
I20260807 15:18:57.957267 26880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:57.958020 26880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:57.959227 26880 hybrid_clock.cc:648] HybridClock initialized: now 1786115937959201 us; error 34 us; skew 500 ppm
I20260807 15:18:57.961284 26880 webserver.cc:460] Webserver started at http://127.31.230.194:34821/ using document root <none> and password file <none>
I20260807 15:18:57.962010 26880 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:57.962114 26880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:57.962360 26880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:57.964525 26880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "10bd706722a0478e845cf480bc48ee1f"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "e635e00ce8f34e9bd498c4e2eb0da658"
server_key_iv: "58a4eae84e847bf4bb8ef1f63760b93d"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.965122 26880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "10bd706722a0478e845cf480bc48ee1f"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "e635e00ce8f34e9bd498c4e2eb0da658"
server_key_iv: "58a4eae84e847bf4bb8ef1f63760b93d"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.969094 26880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:18:57.971607 26899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:57.972816 26880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:18:57.972952 26880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "10bd706722a0478e845cf480bc48ee1f"
format_stamp: "Formatted at 2026-08-07 15:18:57 on dist-test-slave-gkw7"
server_key: "e635e00ce8f34e9bd498c4e2eb0da658"
server_key_iv: "58a4eae84e847bf4bb8ef1f63760b93d"
server_key_version: "encryptionkey@0"
I20260807 15:18:57.973067 26880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:57.990628 26880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:57.991405 26880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:57.991631 26880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:57.992271 26880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:57.993252 26880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:57.993324 26880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:57.993397 26880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:57.993443 26880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:58.002995 26880 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:33545
I20260807 15:18:58.003014 27049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:33545 every 8 connection(s)
I20260807 15:18:58.003961 26880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:18:58.010478 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 26880
I20260807 15:18:58.010661 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:18:58.010957 32667 external_mini_cluster.cc:1428] Setting key cc1fca26c2d964b1feb2eec8c1278c72
I20260807 15:18:58.013492 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37197
--builtin_ntp_servers=127.31.230.212:44193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:18:58.014096 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.005859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:43871 (local address 127.31.230.254:37197)
0807 15:18:58.005992 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.005997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.007235 (+ 1238us) server_negotiation.cc:407] Connection header received
0807 15:18:58.008708 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.008711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.008777 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.008886 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:58.009872 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.010432 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.011882 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.012112 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.012233 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.013579 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:58.013584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:58.013586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:58.013601 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:58.013625 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:58.013634 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:58.013715 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.013889 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.013955 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:18:58.015024 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.006396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37197 (local address 127.31.230.194:43871)
0807 15:18:58.007074 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.007115 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.008476 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.009073 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.009080 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.009173 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:58.009725 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.009740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.010578 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.010581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.011749 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.011759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.012276 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.012280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.012473 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.013251 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:18:58.013253 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:58.013295 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:18:58.013364 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:58.013730 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:58.013748 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:58.013753 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.013888 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.014210 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":135,"threads_started":1}
I20260807 15:18:58.016285 27050 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37197
I20260807 15:18:58.016611 27050 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:58.017421 27050 heartbeater.cc:511] Master 127.31.230.254:37197 requested a full tablet report, sending...
I20260807 15:18:58.018757 26611 ts_manager.cc:194] Registered new tserver with Master: 10bd706722a0478e845cf480bc48ee1f (127.31.230.194:33545)
I20260807 15:18:58.019407 26611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.194:43871
W20260807 15:18:58.137912 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:18:58.138211 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:18:58.138295 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:18:58.142349 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:18:58.142416 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:18:58.142519 27056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:18:58.147294 27056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:44193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:37197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:18:58.148494 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:18:58.150074 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:18:58.160509 27062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:58.160524 27064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:18:58.160522 27067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:18:58.160665 27056 server_base.cc:1034] running on GCE node
I20260807 15:18:58.161281 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:18:58.162047 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:18:58.163242 27056 hybrid_clock.cc:648] HybridClock initialized: now 1786115938163215 us; error 45 us; skew 500 ppm
I20260807 15:18:58.165164 27056 webserver.cc:460] Webserver started at http://127.31.230.195:37439/ using document root <none> and password file <none>
I20260807 15:18:58.165874 27056 fs_manager.cc:359] Metadata directory not provided
I20260807 15:18:58.165969 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:18:58.166208 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:18:58.168303 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "1a64d0a42f444f748d28875af4e3ea04"
format_stamp: "Formatted at 2026-08-07 15:18:58 on dist-test-slave-gkw7"
server_key: "6c67a59b4decdeb6334e7c17b8ac2537"
server_key_iv: "c606f47f5085359aeeeb58375cd3c7d4"
server_key_version: "encryptionkey@0"
I20260807 15:18:58.168890 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "1a64d0a42f444f748d28875af4e3ea04"
format_stamp: "Formatted at 2026-08-07 15:18:58 on dist-test-slave-gkw7"
server_key: "6c67a59b4decdeb6334e7c17b8ac2537"
server_key_iv: "c606f47f5085359aeeeb58375cd3c7d4"
server_key_version: "encryptionkey@0"
I20260807 15:18:58.172775 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:18:58.175308 27074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:58.176563 27056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:18:58.176709 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "1a64d0a42f444f748d28875af4e3ea04"
format_stamp: "Formatted at 2026-08-07 15:18:58 on dist-test-slave-gkw7"
server_key: "6c67a59b4decdeb6334e7c17b8ac2537"
server_key_iv: "c606f47f5085359aeeeb58375cd3c7d4"
server_key_version: "encryptionkey@0"
I20260807 15:18:58.176821 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:18:58.189440 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:18:58.190146 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:18:58.190352 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:18:58.190951 27056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:18:58.191896 27056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:18:58.191970 27056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:58.192042 27056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:18:58.192085 27056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:18:58.201224 27056 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:38349
I20260807 15:18:58.201265 27235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:38349 every 8 connection(s)
I20260807 15:18:58.202179 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:18:58.203150 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 27056
I20260807 15:18:58.203249 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:18:58.203552 32667 external_mini_cluster.cc:1428] Setting key 464d8fb167c6f49c1964563d92860f1d
I20260807 15:18:58.211727 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.204189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:42961 (local address 127.31.230.254:37197)
0807 15:18:58.204347 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.204353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.205660 (+ 1307us) server_negotiation.cc:407] Connection header received
0807 15:18:58.207184 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.207187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.207236 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.207307 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:58.208179 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.208706 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.209976 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.210218 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.210300 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.211318 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:58.211321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:58.211323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:58.211335 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:58.211353 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:58.211361 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:58.211420 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.211558 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.211610 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:18:58.212638 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.204665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:37197 (local address 127.31.230.195:42961)
0807 15:18:58.205474 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.205554 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.206952 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.207445 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.207452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.207537 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:18:58.208037 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.208049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.208812 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.208814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.209862 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.209872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.210355 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.210357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.210505 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.211067 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:18:58.211069 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:18:58.211109 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:18:58.211171 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:18:58.211435 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:18:58.211441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:18:58.211446 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.211540 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.211763 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":162,"threads_started":1}
I20260807 15:18:58.213835 27236 heartbeater.cc:347] Connected to a master server at 127.31.230.254:37197
I20260807 15:18:58.214046 27236 heartbeater.cc:464] Registering TS with master...
I20260807 15:18:58.214512 27236 heartbeater.cc:511] Master 127.31.230.254:37197 requested a full tablet report, sending...
I20260807 15:18:58.215588 26611 ts_manager.cc:194] Registered new tserver with Master: 1a64d0a42f444f748d28875af4e3ea04 (127.31.230.195:38349)
I20260807 15:18:58.216423 26611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.195:42961
I20260807 15:18:58.218004 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:18:58.223652 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.220038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.31.230.254:37197)
0807 15:18:58.220159 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.220193 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.220373 (+ 180us) server_negotiation.cc:407] Connection header received
0807 15:18:58.220470 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.220472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.220509 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.220575 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:18:58.220834 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.221336 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.222237 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.222442 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.222519 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.223292 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:18:58.223295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:18:58.223297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:18:58.223309 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:18:58.223325 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:18:58.223333 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:18:58.223427 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.223524 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.223553 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:18:58.226603 26610 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:18:58.229502 26610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:18:58.244593 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.238520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.31.230.193:39287)
0807 15:18:58.239326 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.239339 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.239368 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:18:58.239462 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.239470 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.239715 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.240179 (+ 464us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:58.240339 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.241740 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.243182 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.243808 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.243968 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.244186 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.244293 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.244395 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":160,"threads_started":1}
I20260807 15:18:58.244587 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.238233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39287 (local address 127.0.0.1:56530)
0807 15:18:58.239195 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.239207 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.239275 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.239938 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.239946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.239974 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:58.240187 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.240195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.241892 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.241895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.243078 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.243087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.243956 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.243960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.244089 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.244132 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.244262 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.244395 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":47,"thread_start_us":97,"threads_started":1}
I20260807 15:18:58.244589 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.239002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:38349 (local address 127.0.0.1:58424)
0807 15:18:58.239703 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.239716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.239807 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.240477 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.240480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.240498 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:58.240571 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.240576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.241501 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.241506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.243046 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.243059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.244152 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.244156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.244257 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.244279 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.244334 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.244410 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":125,"thread_start_us":113,"threads_started":1}
I20260807 15:18:58.245074 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.238208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:33545 (local address 127.0.0.1:60734)
0807 15:18:58.239059 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.239115 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.239275 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.240021 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.240024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.240041 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:58.240316 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.240324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.241930 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.241934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.243121 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.243135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.244388 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.244392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.244470 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.244492 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.244557 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.244626 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":9,"thread_start_us":110,"threads_started":1}
I20260807 15:18:58.245491 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.239614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.31.230.195:38349)
0807 15:18:58.240061 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.240072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.240093 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:18:58.240163 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.240167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.240345 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.240495 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:58.240670 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.241384 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.243210 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.243996 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.245004 (+ 1008us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.245181 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.245235 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.245331 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":160,"threads_started":1}
I20260807 15:18:58.245577 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.238331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.31.230.194:33545)
0807 15:18:58.239101 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.239122 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.239413 (+ 291us) server_negotiation.cc:407] Connection header received
0807 15:18:58.239495 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.239500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.239698 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.239857 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:58.240449 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.241740 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.243240 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.243996 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.244152 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.244326 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.245305 (+ 979us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.245400 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":149,"threads_started":1}
I20260807 15:18:58.247843 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1c3ad18954f7bbc97afba76c3d58f (DEFAULT_TABLE table=test-table [id=ef0e92f1a5c143faace6bbe9881a8e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:58.248193 27140 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1c3ad18954f7bbc97afba76c3d58f (DEFAULT_TABLE table=test-table [id=ef0e92f1a5c143faace6bbe9881a8e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:58.248188 26959 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1c3ad18954f7bbc97afba76c3d58f (DEFAULT_TABLE table=test-table [id=ef0e92f1a5c143faace6bbe9881a8e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:18:58.248970 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1c3ad18954f7bbc97afba76c3d58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:58.249217 27140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1c3ad18954f7bbc97afba76c3d58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:58.249217 26959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1c3ad18954f7bbc97afba76c3d58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:18:58.256548 27258 tablet_bootstrap.cc:492] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Bootstrap starting.
I20260807 15:18:58.257513 27259 tablet_bootstrap.cc:492] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Bootstrap starting.
I20260807 15:18:58.259481 27258 tablet_bootstrap.cc:654] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:58.259857 27259 tablet_bootstrap.cc:654] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:58.260550 27258 log.cc:824] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:58.260727 27259 log.cc:824] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:58.261390 27260 tablet_bootstrap.cc:492] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: Bootstrap starting.
I20260807 15:18:58.263005 27258 tablet_bootstrap.cc:492] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: No bootstrap required, opened a new log
I20260807 15:18:58.263190 27259 tablet_bootstrap.cc:492] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: No bootstrap required, opened a new log
I20260807 15:18:58.263304 27258 ts_tablet_manager.cc:1397] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 15:18:58.263384 27259 ts_tablet_manager.cc:1397] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 15:18:58.263978 27260 tablet_bootstrap.cc:654] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: Neither blocks nor log segments found. Creating new log.
I20260807 15:18:58.264854 27260 log.cc:824] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:18:58.267019 27260 tablet_bootstrap.cc:492] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: No bootstrap required, opened a new log
I20260807 15:18:58.267247 27260 ts_tablet_manager.cc:1397] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:18:58.267045 27258 raft_consensus.cc:348] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.267361 27258 raft_consensus.cc:374] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:58.267649 27258 raft_consensus.cc:729] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10bd706722a0478e845cf480bc48ee1f, State: Initialized, Role: FOLLOWER
I20260807 15:18:58.267705 27259 raft_consensus.cc:348] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.267967 27259 raft_consensus.cc:374] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:58.268026 27259 raft_consensus.cc:729] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a64d0a42f444f748d28875af4e3ea04, State: Initialized, Role: FOLLOWER
I20260807 15:18:58.268186 27258 consensus_queue.cc:260] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.268502 27259 consensus_queue.cc:260] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.269337 27258 ts_tablet_manager.cc:1428] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 15:18:58.269446 27259 ts_tablet_manager.cc:1428] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260807 15:18:58.269774 27050 heartbeater.cc:503] Master 127.31.230.254:37197 was elected leader, sending a full tablet report...
I20260807 15:18:58.269809 27236 heartbeater.cc:503] Master 127.31.230.254:37197 was elected leader, sending a full tablet report...
I20260807 15:18:58.271759 27260 raft_consensus.cc:348] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.271948 27260 raft_consensus.cc:374] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:18:58.271981 27260 raft_consensus.cc:729] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2022749872aa41a5b0e007adaf23dfe1, State: Initialized, Role: FOLLOWER
I20260807 15:18:58.272529 27260 consensus_queue.cc:260] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.273368 26875 heartbeater.cc:503] Master 127.31.230.254:37197 was elected leader, sending a full tablet report...
I20260807 15:18:58.273744 27260 ts_tablet_manager.cc:1428] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 15:18:58.301599 26878 tablet.cc:2369] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:18:58.453159 27237 tablet.cc:2369] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:18:58.462028 27265 raft_consensus.cc:482] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:18:58.462312 27265 raft_consensus.cc:504] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.463547 27265 leader_election.cc:288] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2022749872aa41a5b0e007adaf23dfe1 (127.31.230.193:39287), 1a64d0a42f444f748d28875af4e3ea04 (127.31.230.195:38349)
I20260807 15:18:58.467389 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.464040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:57835 (local address 127.31.230.195:38349)
0807 15:18:58.464165 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.464170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.464185 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:18:58.464301 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.464304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.464354 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.464444 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:58.464751 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.465270 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.466408 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.466805 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.466938 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.466992 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.467181 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.467260 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:18:58.467389 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.463885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:38349 (local address 127.31.230.194:57835)
0807 15:18:58.464048 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.464068 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.464190 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.464461 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.464464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.464483 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:58.464609 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.464616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.465391 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.465395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.466267 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.466277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.466937 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.466940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.467043 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.467071 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.467162 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.467246 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:18:58.467662 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.464019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:55531 (local address 127.31.230.193:39287)
0807 15:18:58.464163 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.464169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.464560 (+ 391us) server_negotiation.cc:407] Connection header received
0807 15:18:58.464726 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.464730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.464781 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.464871 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:18:58.465149 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.465693 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.466684 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.467085 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.467237 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.467287 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.467472 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.467543 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:18:58.467682 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.463885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39287 (local address 127.31.230.194:55531)
0807 15:18:58.464452 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.464464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:18:58.464597 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:18:58.464882 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:18:58.464885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:18:58.464901 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:18:58.465028 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.465034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.465807 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.465812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.466554 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:18:58.466562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.467237 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.467241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:18:58.467337 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.467363 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:18:58.467453 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:18:58.467524 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":161,"thread_start_us":188,"threads_started":1}
I20260807 15:18:58.468406 26810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1c3ad18954f7bbc97afba76c3d58f" candidate_uuid: "10bd706722a0478e845cf480bc48ee1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2022749872aa41a5b0e007adaf23dfe1" is_pre_election: true
I20260807 15:18:58.468408 27172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1c3ad18954f7bbc97afba76c3d58f" candidate_uuid: "10bd706722a0478e845cf480bc48ee1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a64d0a42f444f748d28875af4e3ea04" is_pre_election: true
I20260807 15:18:58.468752 26810 raft_consensus.cc:2454] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10bd706722a0478e845cf480bc48ee1f in term 0.
I20260807 15:18:58.468752 27172 raft_consensus.cc:2454] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10bd706722a0478e845cf480bc48ee1f in term 0.
I20260807 15:18:58.469340 26900 leader_election.cc:302] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10bd706722a0478e845cf480bc48ee1f, 1a64d0a42f444f748d28875af4e3ea04, 2022749872aa41a5b0e007adaf23dfe1; no voters:
I20260807 15:18:58.469700 27265 raft_consensus.cc:2781] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:18:58.469827 27265 raft_consensus.cc:482] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:18:58.469906 27265 raft_consensus.cc:3037] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:58.471030 27265 raft_consensus.cc:504] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.471407 27265 leader_election.cc:288] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [CANDIDATE]: Term 1 election: Requested vote from peers 2022749872aa41a5b0e007adaf23dfe1 (127.31.230.193:39287), 1a64d0a42f444f748d28875af4e3ea04 (127.31.230.195:38349)
I20260807 15:18:58.471891 26810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1c3ad18954f7bbc97afba76c3d58f" candidate_uuid: "10bd706722a0478e845cf480bc48ee1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2022749872aa41a5b0e007adaf23dfe1"
I20260807 15:18:58.471891 27172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1c3ad18954f7bbc97afba76c3d58f" candidate_uuid: "10bd706722a0478e845cf480bc48ee1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a64d0a42f444f748d28875af4e3ea04"
I20260807 15:18:58.472051 26810 raft_consensus.cc:3037] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:58.472039 27172 raft_consensus.cc:3037] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:18:58.473565 26810 raft_consensus.cc:2454] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10bd706722a0478e845cf480bc48ee1f in term 1.
I20260807 15:18:58.473559 27172 raft_consensus.cc:2454] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10bd706722a0478e845cf480bc48ee1f in term 1.
I20260807 15:18:58.473894 26900 leader_election.cc:302] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10bd706722a0478e845cf480bc48ee1f, 2022749872aa41a5b0e007adaf23dfe1; no voters:
I20260807 15:18:58.474097 27265 raft_consensus.cc:2781] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:18:58.474362 27265 raft_consensus.cc:686] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 1 LEADER]: Becoming Leader. State: Replica: 10bd706722a0478e845cf480bc48ee1f, State: Running, Role: LEADER
I20260807 15:18:58.474699 27265 consensus_queue.cc:237] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } }
I20260807 15:18:58.478231 26611 catalog_manager.cc:5495] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 10bd706722a0478e845cf480bc48ee1f (127.31.230.194). New cstate: current_term: 1 leader_uuid: "10bd706722a0478e845cf480bc48ee1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bd706722a0478e845cf480bc48ee1f" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 33545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 } health_report { overall_health: UNKNOWN } } }
W20260807 15:18:58.505395 27051 tablet.cc:2369] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:18:58.540230 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.536500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.31.230.194:33545)
0807 15:18:58.536650 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:18:58.536654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:18:58.536672 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:18:58.536772 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:18:58.536774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:18:58.536833 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:18:58.536950 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:18:58.537227 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.537774 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.538675 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:18:58.538902 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:18:58.539018 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:18:58.539335 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:18:58.539726 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:18:58.539893 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:18:58.539990 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:18:58.540028 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:18:58.546180 27172 raft_consensus.cc:1261] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 1 FOLLOWER]: Refusing update from remote peer 10bd706722a0478e845cf480bc48ee1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:58.546180 26810 raft_consensus.cc:1261] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 10bd706722a0478e845cf480bc48ee1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:18:58.546950 27265 consensus_queue.cc:1035] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a64d0a42f444f748d28875af4e3ea04" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 38349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:58.547156 27273 consensus_queue.cc:1035] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2022749872aa41a5b0e007adaf23dfe1" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:18:58.554147 27276 mvcc.cc:205] Tried to move back new op lower bound from 7315930884277440512 to 7315930883994996736. Current Snapshot: MvccSnapshot[applied={T|T < 7315930884277440512}]
I20260807 15:18:58.563731 26610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:58.563920 26610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:18:58.566460 26610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0635836939c944b8be8579877d6ed71c: Sending DeleteTablet for 3 replicas of tablet 44c1c3ad18954f7bbc97afba76c3d58f
I20260807 15:18:58.567269 26959 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1c3ad18954f7bbc97afba76c3d58f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:58 UTC) from {username='slave'} at 127.0.0.1:60734
I20260807 15:18:58.567288 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1c3ad18954f7bbc97afba76c3d58f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:58 UTC) from {username='slave'} at 127.0.0.1:56530
I20260807 15:18:58.567797 27287 tablet_replica.cc:330] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: stopping tablet replica
I20260807 15:18:58.567814 27288 tablet_replica.cc:330] T 44c1c3ad18954f7bbc97afba76c3d58f P 2022749872aa41a5b0e007adaf23dfe1: stopping tablet replica
I20260807 15:18:58.568019 27140 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1c3ad18954f7bbc97afba76c3d58f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:18:58 UTC) from {username='slave'} at 127.0.0.1:58424
I20260807 15:18:58.568125 27287 raft_consensus.cc:2229] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:18:58.568477 27287 raft_consensus.cc:2258] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:58.568738 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 26695
I20260807 15:18:58.568733 27291 tablet_replica.cc:330] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: stopping tablet replica
I20260807 15:18:58.569110 27291 raft_consensus.cc:2229] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:18:58.569630 27291 raft_consensus.cc:2258] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:18:58.572664 27291 ts_tablet_manager.cc:1905] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:58.576117 27291 ts_tablet_manager.cc:1918] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:58.576241 27291 log.cc:1196] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/44c1c3ad18954f7bbc97afba76c3d58f
I20260807 15:18:58.576536 27291 ts_tablet_manager.cc:1939] T 44c1c3ad18954f7bbc97afba76c3d58f P 1a64d0a42f444f748d28875af4e3ea04: Deleting consensus metadata
I20260807 15:18:58.577919 26595 catalog_manager.cc:4841] TS 1a64d0a42f444f748d28875af4e3ea04 (127.31.230.195:38349): tablet 44c1c3ad18954f7bbc97afba76c3d58f (table test-table [id=ef0e92f1a5c143faace6bbe9881a8e01]) successfully deleted
I20260807 15:18:58.578644 27287 ts_tablet_manager.cc:1905] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:18:58.581161 27287 ts_tablet_manager.cc:1918] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:18:58.581275 27287 log.cc:1196] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/44c1c3ad18954f7bbc97afba76c3d58f
I20260807 15:18:58.581616 27287 ts_tablet_manager.cc:1939] T 44c1c3ad18954f7bbc97afba76c3d58f P 10bd706722a0478e845cf480bc48ee1f: Deleting consensus metadata
I20260807 15:18:58.582682 26595 catalog_manager.cc:4841] TS 10bd706722a0478e845cf480bc48ee1f (127.31.230.194:33545): tablet 44c1c3ad18954f7bbc97afba76c3d58f (table test-table [id=ef0e92f1a5c143faace6bbe9881a8e01]) successfully deleted
W20260807 15:18:58.583151 26593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:39287 (error 108)
I20260807 15:18:58.583415 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 26880
I20260807 15:18:58.585882 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:18:58.585485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:39287 (local address 127.0.0.1:56538)
0807 15:18:58.585697 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 15:18:58.585784 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260807 15:18:58.586072 26593 catalog_manager.cc:4584] TS 2022749872aa41a5b0e007adaf23dfe1 (127.31.230.193:39287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44c1c3ad18954f7bbc97afba76c3d58f: Network error: Client connection negotiation failed: client connection to 127.31.230.193:39287: connect: Connection refused (error 111)
I20260807 15:18:58.606686 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 27056
I20260807 15:18:58.621426 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 26569
2026-08-07T15:18:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1315 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:18:58 dist-test-slave-gkw7 krb5kdc[27296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:18:58 dist-test-slave-gkw7 krb5kdc[27296](info): set up 2 sockets
Aug 07 15:18:58 dist-test-slave-gkw7 krb5kdc[27296](info): commencing operation
krb5kdc: starting...
W20260807 15:18:59.851217 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:18:59 dist-test-slave-gkw7 krb5kdc[27296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:18:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:18:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:18:59.990592 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45469
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:00.108830 27319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:00.109154 27319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:00.109243 27319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:00.113224 27319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:00.113317 27319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:00.113358 27319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:00.113394 27319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:00.113430 27319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:00.118791 27319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:00.119916 27319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:00.120975 27319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:19:00.130280 27319 server_base.cc:1034] running on GCE node
W20260807 15:19:00.130312 27328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:00.130336 27325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:00.130431 27326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:00.131047 27319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:00.132222 27319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:00.133417 27319 hybrid_clock.cc:648] HybridClock initialized: now 1786115940133386 us; error 42 us; skew 500 ppm
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:00.136399 27319 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:00.137363 27319 webserver.cc:460] Webserver started at http://127.31.230.254:42999/ using document root <none> and password file <none>
I20260807 15:19:00.138010 27319 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:00.138082 27319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:00.138330 27319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:00.140407 27319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "7218d1cd356746a5a1e01b09c9fd3c1b"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "86d9517c4b889f1551e1cc5df0963444"
server_key_iv: "1f2e0281d044e3970b9467cafaa07a83"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.140993 27319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "7218d1cd356746a5a1e01b09c9fd3c1b"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "86d9517c4b889f1551e1cc5df0963444"
server_key_iv: "1f2e0281d044e3970b9467cafaa07a83"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.144851 27319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:19:00.147403 27339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.148648 27319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:19:00.148790 27319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "7218d1cd356746a5a1e01b09c9fd3c1b"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "86d9517c4b889f1551e1cc5df0963444"
server_key_iv: "1f2e0281d044e3970b9467cafaa07a83"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.148903 27319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:00.169030 27319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:00.172492 27319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:00.172731 27319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:00.180352 27319 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:45469
I20260807 15:19:00.180382 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:45469 every 8 connection(s)
I20260807 15:19:00.181346 27319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:00.184221 27412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:00.188588 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 27319
I20260807 15:19:00.188750 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:00.189028 32667 external_mini_cluster.cc:1428] Setting key acf37b5661a2b53f7bcbe677dabc1e6e
I20260807 15:19:00.190297 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b: Bootstrap starting.
I20260807 15:19:00.193060 27412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:00.193960 27412 log.cc:824] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:00.196067 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b: No bootstrap required, opened a new log
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:00.200214 27412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45469 } }
I20260807 15:19:00.200528 27412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:00.200606 27412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7218d1cd356746a5a1e01b09c9fd3c1b, State: Initialized, Role: FOLLOWER
I20260807 15:19:00.201119 27412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45469 } }
I20260807 15:19:00.201299 27412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:00.201372 27412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:00.201478 27412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:00.202750 27412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45469 } }
I20260807 15:19:00.203125 27412 leader_election.cc:302] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7218d1cd356746a5a1e01b09c9fd3c1b; no voters:
I20260807 15:19:00.203526 27412 leader_election.cc:288] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:00.203603 27419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:00.203958 27419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [term 1 LEADER]: Becoming Leader. State: Replica: 7218d1cd356746a5a1e01b09c9fd3c1b, State: Running, Role: LEADER
I20260807 15:19:00.204263 27419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45469 } }
I20260807 15:19:00.204878 27412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:00.204991 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.190604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.31.230.254:45469)
0807 15:19:00.191016 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:00.191034 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:00.191077 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:19:00.191950 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:00.191974 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:00.192388 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:00.192756 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:00.193552 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.194956 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.195881 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.196541 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.196728 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.199287 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:00.199295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:00.199315 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:00.199347 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:00.201407 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.201965 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.201972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.201977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.202118 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.202438 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.202441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.202443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.202997 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:00.203197 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:00.203491 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 15:19:00.203961 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":108,"threads_started":1}
I20260807 15:19:00.205514 27419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7218d1cd356746a5a1e01b09c9fd3c1b. Latest consensus state: current_term: 1 leader_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45469 } } }
I20260807 15:19:00.205703 27419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:00.206086 27421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218d1cd356746a5a1e01b09c9fd3c1b" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 45469 } } }
I20260807 15:19:00.206250 27421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:00.207701 27428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:00.210456 27428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:00.216564 27428 catalog_manager.cc:1347] Generated new cluster ID: 82f0b946991048c2b412269be00f5638
I20260807 15:19:00.216636 27428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:00.225394 27428 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:00.226691 27428 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:00.234877 27428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b: Generated new TSK 0
I20260807 15:19:00.235718 27428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.230.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
Entry for principal kudu/127.31.230.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab.
I20260807 15:19:00.269223 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45469
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:00.392359 27443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:00.392633 27443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:00.392678 27443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:00.396724 27443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:00.396785 27443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:00.396842 27443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:19:00.402122 27443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.193.keytab
--principal=kudu/127.31.230.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:00.403133 27443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:00.404160 27443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:00.414088 27449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:00.414124 27452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:00.414132 27450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:00.414381 27443 server_base.cc:1034] running on GCE node
I20260807 15:19:00.414821 27443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:00.415457 27443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:00.416678 27443 hybrid_clock.cc:648] HybridClock initialized: now 1786115940416652 us; error 41 us; skew 500 ppm
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:00.419749 27443 init.cc:377] Logged in from keytab as kudu/127.31.230.193@KRBTEST.COM (short username kudu)
I20260807 15:19:00.420636 27443 webserver.cc:460] Webserver started at http://127.31.230.193:32859/ using document root <none> and password file <none>
I20260807 15:19:00.421269 27443 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:00.421368 27443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:00.421676 27443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:00.423909 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "d74eab887be34ea2a4a971fb770aa11a"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "97a1c76016774643be5ce8fdef3364f3"
server_key_iv: "0e2cfcc331f61e6a9fde955a296e74dc"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.424582 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "d74eab887be34ea2a4a971fb770aa11a"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "97a1c76016774643be5ce8fdef3364f3"
server_key_iv: "0e2cfcc331f61e6a9fde955a296e74dc"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.428561 27443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:19:00.431351 27465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.432673 27443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:19:00.432808 27443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "d74eab887be34ea2a4a971fb770aa11a"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "97a1c76016774643be5ce8fdef3364f3"
server_key_iv: "0e2cfcc331f61e6a9fde955a296e74dc"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.432920 27443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:00.461895 27443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:00.465481 27443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:00.465864 27443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:00.466588 27443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:19:00.467581 27443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:19:00.467634 27443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.467710 27443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:19:00.467747 27443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.477949 27443 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:38469
I20260807 15:19:00.477977 27612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:38469 every 8 connection(s)
I20260807 15:19:00.479035 27443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:19:00.488682 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 27443
I20260807 15:19:00.488848 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:19:00.489178 32667 external_mini_cluster.cc:1428] Setting key bd8bed4a3c5d6c699476c2d7c5194ed9
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.193@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:00.494465 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.481143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:36581 (local address 127.31.230.254:45469)
0807 15:19:00.481335 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:00.481339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:00.482211 (+ 872us) server_negotiation.cc:407] Connection header received
0807 15:19:00.483588 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:00.483591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:00.483651 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:00.483761 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:00.484977 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.485602 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.486934 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.487191 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.487309 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.490333 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:00.490337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:00.490339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:00.490365 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:00.492131 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.492667 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.492673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.492678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.492782 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.493121 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.493126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.493128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.493390 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:00.493491 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:00.494226 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 15:19:00.494339 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 15:19:00.495810 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.481516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45469 (local address 127.31.230.193:36581)
0807 15:19:00.482047 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:00.482091 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:00.483361 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:00.483939 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:00.483946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:00.484321 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:00.484816 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:00.484830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.485742 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.485745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:00.486797 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:00.486807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.487356 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.487360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:00.487529 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.488185 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:00.488202 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:00.490124 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:00.492273 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:00.492278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:00.492292 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:00.492557 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:00.492882 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:00.492884 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:00.492886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:00.493025 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:00.493559 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:00.493570 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:00.493931 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 15:19:00.494216 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 15:19:00.494627 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":130,"threads_started":1}
I20260807 15:19:00.497383 27614 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45469
I20260807 15:19:00.497716 27614 heartbeater.cc:464] Registering TS with master...
I20260807 15:19:00.498260 27614 heartbeater.cc:511] Master 127.31.230.254:45469 requested a full tablet report, sending...
I20260807 15:19:00.500070 27360 ts_manager.cc:194] Registered new tserver with Master: d74eab887be34ea2a4a971fb770aa11a (127.31.230.193:38469)
WARNING: no policy specified for kudu/127.31.230.194@KRBTEST.COM; defaulting to no policy
I20260807 15:19:00.501839 27360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.193@KRBTEST.COM'} at 127.31.230.193:36581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
Entry for principal kudu/127.31.230.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab.
I20260807 15:19:00.521795 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45469
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:00.640735 27620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:00.641044 27620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:00.641122 27620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:00.645263 27620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:00.645370 27620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:00.645466 27620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:19:00.650939 27620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.194.keytab
--principal=kudu/127.31.230.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:00.652107 27620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:00.653240 27620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:00.663892 27627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:00.663892 27626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:00.664037 27620 server_base.cc:1034] running on GCE node
W20260807 15:19:00.664243 27629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:00.664669 27620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:00.665396 27620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:00.666606 27620 hybrid_clock.cc:648] HybridClock initialized: now 1786115940666578 us; error 38 us; skew 500 ppm
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:00.669836 27620 init.cc:377] Logged in from keytab as kudu/127.31.230.194@KRBTEST.COM (short username kudu)
I20260807 15:19:00.670871 27620 webserver.cc:460] Webserver started at http://127.31.230.194:40509/ using document root <none> and password file <none>
I20260807 15:19:00.671537 27620 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:00.671639 27620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:00.671900 27620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:00.674137 27620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "09629248a1db4e6c9d5436f0fe7d1abb"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "59f86c9de69e0a31ab43abae005f3d6c"
server_key_iv: "2b50bbc796878bd4b82b5e010e3d8024"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.674729 27620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "09629248a1db4e6c9d5436f0fe7d1abb"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "59f86c9de69e0a31ab43abae005f3d6c"
server_key_iv: "2b50bbc796878bd4b82b5e010e3d8024"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.678627 27620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:19:00.681164 27636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.682549 27620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:19:00.682694 27620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "09629248a1db4e6c9d5436f0fe7d1abb"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "59f86c9de69e0a31ab43abae005f3d6c"
server_key_iv: "2b50bbc796878bd4b82b5e010e3d8024"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.682811 27620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:00.713982 27620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:00.717519 27620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:00.717792 27620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:00.718448 27620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:19:00.719331 27620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:19:00.719403 27620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.719478 27620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:19:00.719532 27620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.729000 27620 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:43085
I20260807 15:19:00.729019 27796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:43085 every 8 connection(s)
I20260807 15:19:00.730006 27620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.194@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:00.740135 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 27620
I20260807 15:19:00.740278 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:19:00.740561 32667 external_mini_cluster.cc:1428] Setting key 73d246b7ccb4201b816981842a751746
I20260807 15:19:00.744930 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.731953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:43141 (local address 127.31.230.254:45469)
0807 15:19:00.732105 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:00.732109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:00.732938 (+ 829us) server_negotiation.cc:407] Connection header received
0807 15:19:00.734401 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:00.734405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:00.734464 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:00.734571 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:00.735804 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.736440 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.737680 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.737913 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.738024 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.740828 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:00.740832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:00.740835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:00.740865 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:00.742719 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.743232 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.743239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.743243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.743345 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.743682 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.743688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.743693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.743960 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:00.744092 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:00.744597 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 15:19:00.744753 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:19:00.745888 27802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.732341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45469 (local address 127.31.230.194:43141)
0807 15:19:00.732776 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:00.732822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:00.734176 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:00.734749 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:00.734756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:00.735157 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:00.735654 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:00.735667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.736573 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.736577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:00.737522 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:00.737538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.738092 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.738098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:00.738261 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.738835 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:00.738852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:00.740617 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:00.742842 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:00.742847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:00.742861 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:00.743117 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:00.743443 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:00.743446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:00.743447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:00.743586 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:00.744078 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:00.744083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:00.744344 (+ 261us) client_negotiation.cc:769] Sending connection context
0807 15:19:00.744551 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 15:19:00.744895 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":101,"threads_started":1}
I20260807 15:19:00.747076 27797 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45469
I20260807 15:19:00.747375 27797 heartbeater.cc:464] Registering TS with master...
I20260807 15:19:00.748029 27797 heartbeater.cc:511] Master 127.31.230.254:45469 requested a full tablet report, sending...
I20260807 15:19:00.749289 27360 ts_manager.cc:194] Registered new tserver with Master: 09629248a1db4e6c9d5436f0fe7d1abb (127.31.230.194:43085)
I20260807 15:19:00.750201 27360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.194@KRBTEST.COM'} at 127.31.230.194:43141
WARNING: no policy specified for kudu/127.31.230.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
Entry for principal kudu/127.31.230.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab.
I20260807 15:19:00.772454 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45469
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:00.888993 27807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:00.889267 27807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:00.889312 27807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:00.893323 27807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:00.893385 27807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:00.893472 27807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:19:00.898890 27807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.195.keytab
--principal=kudu/127.31.230.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:00.900002 27807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:00.901114 27807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:00.911413 27814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:00.911423 27817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:00.911491 27807 server_base.cc:1034] running on GCE node
W20260807 15:19:00.911423 27815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:00.912163 27807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:00.912931 27807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:00.914103 27807 hybrid_clock.cc:648] HybridClock initialized: now 1786115940914083 us; error 30 us; skew 500 ppm
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:00.917426 27807 init.cc:377] Logged in from keytab as kudu/127.31.230.195@KRBTEST.COM (short username kudu)
I20260807 15:19:00.918431 27807 webserver.cc:460] Webserver started at http://127.31.230.195:45129/ using document root <none> and password file <none>
I20260807 15:19:00.919023 27807 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:00.919121 27807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:00.919371 27807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:00.921491 27807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "72f2a11167494aa7a6b15d6109d1d6b5"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "98165756e6076ea5d9c8c60da8b3a16c"
server_key_iv: "d56a7ed92b52afbd1a6f78bb83b20867"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.922227 27807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "72f2a11167494aa7a6b15d6109d1d6b5"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "98165756e6076ea5d9c8c60da8b3a16c"
server_key_iv: "d56a7ed92b52afbd1a6f78bb83b20867"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.926551 27807 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:19:00.929658 27830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.931048 27807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:19:00.931200 27807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "72f2a11167494aa7a6b15d6109d1d6b5"
format_stamp: "Formatted at 2026-08-07 15:19:00 on dist-test-slave-gkw7"
server_key: "98165756e6076ea5d9c8c60da8b3a16c"
server_key_iv: "d56a7ed92b52afbd1a6f78bb83b20867"
server_key_version: "encryptionkey@0"
I20260807 15:19:00.931321 27807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:00.961001 27807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:00.964289 27807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:00.964512 27807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:00.965142 27807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:19:00.966045 27807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:19:00.966120 27807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.966205 27807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:19:00.966250 27807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:00.975551 27807 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:46335
I20260807 15:19:00.975579 27982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:46335 every 8 connection(s)
I20260807 15:19:00.976527 27807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:19:00.980422 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 27807
I20260807 15:19:00.980583 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:19:00.980955 32667 external_mini_cluster.cc:1428] Setting key b23c7d7ccc2d448ff3e2ec2782998b46
Aug 07 15:19:00 dist-test-slave-gkw7 krb5kdc[27296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.195@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:00.990700 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.978749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:48361 (local address 127.31.230.254:45469)
0807 15:19:00.978926 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:00.978930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:00.979743 (+ 813us) server_negotiation.cc:407] Connection header received
0807 15:19:00.981240 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:00.981244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:00.981304 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:00.981391 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:00.982620 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.983161 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.984348 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.984581 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.984694 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.987244 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:00.987248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:00.987255 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:00.987300 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:00.988569 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.989092 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.989095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.989098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.989188 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:00.989565 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:00.989569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:00.989571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:00.989764 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:00.989876 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:00.990374 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 15:19:00.990501 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:19:00.991876 27990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:00.979109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:45469 (local address 127.31.230.195:48361)
0807 15:19:00.979565 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:00.979609 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:00.980984 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:00.981593 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:00.981602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:00.981983 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:00.982474 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:00.982487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.983299 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.983303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:00.984218 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:00.984228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:00.984743 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:00.984747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:00.984928 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:00.985486 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:00.985503 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:00.987066 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:00.988697 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:00.988703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:00.988719 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:00.988973 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:00.989280 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:00.989283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:00.989285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:00.989428 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:00.989873 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:00.989880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:00.990134 (+ 254us) client_negotiation.cc:769] Sending connection context
0807 15:19:00.990333 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 15:19:00.990667 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260807 15:19:00.993117 27988 heartbeater.cc:347] Connected to a master server at 127.31.230.254:45469
I20260807 15:19:00.993409 27988 heartbeater.cc:464] Registering TS with master...
I20260807 15:19:00.994316 27988 heartbeater.cc:511] Master 127.31.230.254:45469 requested a full tablet report, sending...
I20260807 15:19:00.995517 27360 ts_manager.cc:194] Registered new tserver with Master: 72f2a11167494aa7a6b15d6109d1d6b5 (127.31.230.195:46335)
I20260807 15:19:00.996122 27360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.230.195@KRBTEST.COM'} at 127.31.230.195:48361
I20260807 15:19:01.007754 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:19:01.017853 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.010156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.31.230.254:45469)
0807 15:19:01.010331 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.010335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.010461 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:19:01.010572 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.010574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.010624 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.010687 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:01.011569 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.012117 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.012940 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.013163 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.013245 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.014672 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:01.014676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:01.014678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:01.014703 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:01.016297 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:01.016694 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:01.016697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:01.016698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:01.016777 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:01.017010 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:01.017012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:01.017013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:01.017235 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:01.017347 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.017603 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.017725 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 15:19:01.021023 27360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:19:01.024169 27360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:19:01.039547 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.032661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58804 (local address 127.31.230.193:38469)
0807 15:19:01.033960 (+ 1299us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.033969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.033991 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:19:01.034069 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.034076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.034288 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.034526 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:01.034804 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.035973 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.037484 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.038392 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.038531 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.038668 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.039250 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.039362 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":116,"threads_started":1}
I20260807 15:19:01.039599 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.032494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38469 (local address 127.0.0.1:58804)
0807 15:19:01.033190 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.033246 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:01.033398 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:01.034434 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:01.034437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:01.034450 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:01.034646 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.034652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.036114 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.036117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.037362 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.037373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.038567 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.038573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.038986 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.039045 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 15:19:01.039197 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 15:19:01.039386 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":271,"spinlock_wait_cycles":3584,"thread_start_us":79,"threads_started":1}
I20260807 15:19:01.040161 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.032759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:46335 (local address 127.0.0.1:42648)
0807 15:19:01.033191 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.033245 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:01.033400 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:01.034311 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:01.034317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:01.034344 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:01.034537 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.034545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.036310 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.036315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.037512 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.037519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.039767 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.039771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.039861 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.039882 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:19:01.039957 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:19:01.040032 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":62,"spinlock_wait_cycles":6144,"thread_start_us":95,"threads_started":1}
I20260807 15:19:01.040400 28011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.033325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.31.230.195:46335)
0807 15:19:01.033803 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.033816 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.033844 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:19:01.033925 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.033932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.034141 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.034307 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:01.034677 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.035960 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.038742 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.039410 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.039529 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.039690 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.040085 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.040218 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":205,"threads_started":1}
I20260807 15:19:01.040711 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.032494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:43085 (local address 127.0.0.1:37414)
0807 15:19:01.033789 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.033803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:01.033899 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:01.034531 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:01.034534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:01.034556 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:01.034657 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.034662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.035784 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.035789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.037493 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.037505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.038567 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.038573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.038990 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.039045 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 15:19:01.039197 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 15:19:01.040542 (+ 1345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"mutex_wait_us":1973,"spinlock_wait_cycles":1792,"thread_start_us":105,"threads_started":1}
I20260807 15:19:01.042017 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.032747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.31.230.194:43085)
0807 15:19:01.033766 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.033778 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.033991 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:19:01.034063 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.034067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.034240 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.034365 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:01.034832 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.035643 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.037621 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.038392 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.038531 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.038668 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.041826 (+ 3158us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.041913 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":128,"threads_started":1}
I20260807 15:19:01.043399 27530 tablet_service.cc:1467] Processing CreateTablet for tablet f8e8e07882704df79d34a35a89ee51e3 (DEFAULT_TABLE table=test-table [id=4c2b5f1dc25c4754ab710e9adf76daf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:19:01.043706 27896 tablet_service.cc:1467] Processing CreateTablet for tablet f8e8e07882704df79d34a35a89ee51e3 (DEFAULT_TABLE table=test-table [id=4c2b5f1dc25c4754ab710e9adf76daf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:19:01.044272 27707 tablet_service.cc:1467] Processing CreateTablet for tablet f8e8e07882704df79d34a35a89ee51e3 (DEFAULT_TABLE table=test-table [id=4c2b5f1dc25c4754ab710e9adf76daf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:19:01.044754 27896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e8e07882704df79d34a35a89ee51e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:01.044754 27530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e8e07882704df79d34a35a89ee51e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:01.045238 27707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e8e07882704df79d34a35a89ee51e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:01.053944 28013 tablet_bootstrap.cc:492] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Bootstrap starting.
I20260807 15:19:01.054768 28015 tablet_bootstrap.cc:492] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Bootstrap starting.
I20260807 15:19:01.053967 28014 tablet_bootstrap.cc:492] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: Bootstrap starting.
I20260807 15:19:01.056502 28013 tablet_bootstrap.cc:654] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:01.057184 28014 tablet_bootstrap.cc:654] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:01.057922 28013 log.cc:824] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:01.057978 28014 log.cc:824] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:01.059731 28015 tablet_bootstrap.cc:654] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:01.060181 28013 tablet_bootstrap.cc:492] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: No bootstrap required, opened a new log
I20260807 15:19:01.060189 28014 tablet_bootstrap.cc:492] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: No bootstrap required, opened a new log
I20260807 15:19:01.060388 28014 ts_tablet_manager.cc:1397] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 15:19:01.060389 28013 ts_tablet_manager.cc:1397] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:19:01.061183 28015 log.cc:824] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:01.064608 28013 raft_consensus.cc:348] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.064926 28013 raft_consensus.cc:374] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:01.064977 28013 raft_consensus.cc:729] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09629248a1db4e6c9d5436f0fe7d1abb, State: Initialized, Role: FOLLOWER
I20260807 15:19:01.064848 28014 raft_consensus.cc:348] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.065143 28014 raft_consensus.cc:374] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:01.065213 28014 raft_consensus.cc:729] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74eab887be34ea2a4a971fb770aa11a, State: Initialized, Role: FOLLOWER
I20260807 15:19:01.065512 28013 consensus_queue.cc:260] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.065670 28014 consensus_queue.cc:260] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.066339 27797 heartbeater.cc:503] Master 127.31.230.254:45469 was elected leader, sending a full tablet report...
I20260807 15:19:01.066635 28014 ts_tablet_manager.cc:1428] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:19:01.066635 28013 ts_tablet_manager.cc:1428] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 15:19:01.066967 27614 heartbeater.cc:503] Master 127.31.230.254:45469 was elected leader, sending a full tablet report...
I20260807 15:19:01.067557 28015 tablet_bootstrap.cc:492] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: No bootstrap required, opened a new log
I20260807 15:19:01.067840 28015 ts_tablet_manager.cc:1397] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20260807 15:19:01.072589 28015 raft_consensus.cc:348] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.073014 28015 raft_consensus.cc:374] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:01.073117 28015 raft_consensus.cc:729] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f2a11167494aa7a6b15d6109d1d6b5, State: Initialized, Role: FOLLOWER
I20260807 15:19:01.073714 28015 consensus_queue.cc:260] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.074589 27988 heartbeater.cc:503] Master 127.31.230.254:45469 was elected leader, sending a full tablet report...
I20260807 15:19:01.075024 28015 ts_tablet_manager.cc:1428] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
W20260807 15:19:01.227589 27989 tablet.cc:2369] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:19:01.230499 27615 tablet.cc:2369] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:19:01.231122 27800 tablet.cc:2369] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:19:01.254585 28020 raft_consensus.cc:482] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:19:01.254832 28020 raft_consensus.cc:504] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.256065 28020 leader_election.cc:288] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d74eab887be34ea2a4a971fb770aa11a (127.31.230.193:38469), 72f2a11167494aa7a6b15d6109d1d6b5 (127.31.230.195:46335)
I20260807 15:19:01.261929 28011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.256517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:59073 (local address 127.31.230.195:46335)
0807 15:19:01.256737 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.256742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.256834 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:19:01.257002 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.257006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.257061 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.257151 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:01.257606 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.258424 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.260396 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.261027 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.261160 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.261236 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.261683 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.261789 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260807 15:19:01.261929 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.256502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:48861 (local address 127.31.230.193:38469)
0807 15:19:01.256704 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.256710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.256835 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:19:01.257002 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.257006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.257061 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.257151 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:01.257606 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.258409 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.260395 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.261039 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.261160 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.261220 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.261683 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.261789 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 15:19:01.261996 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.256350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38469 (local address 127.31.230.194:48861)
0807 15:19:01.256709 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.256725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:01.256884 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:01.257180 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:01.257183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:01.257210 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:01.257459 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.257466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.258553 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.258557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.260247 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.260261 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.261295 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.261299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.261464 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.261549 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 15:19:01.261636 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:19:01.261795 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":281,"spinlock_wait_cycles":18432,"thread_start_us":120,"threads_started":1}
I20260807 15:19:01.262004 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.256411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:46335 (local address 127.31.230.194:59073)
0807 15:19:01.256740 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.256753 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:01.256884 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:01.257180 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:01.257183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:01.257210 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:01.257459 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.257466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.258553 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.258557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.260247 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.260261 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.261295 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.261299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.261461 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.261549 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 15:19:01.261636 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:19:01.261795 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":297,"spinlock_wait_cycles":24832,"thread_start_us":89,"threads_started":1}
I20260807 15:19:01.262852 27553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e8e07882704df79d34a35a89ee51e3" candidate_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74eab887be34ea2a4a971fb770aa11a" is_pre_election: true
I20260807 15:19:01.263063 27923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e8e07882704df79d34a35a89ee51e3" candidate_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" is_pre_election: true
I20260807 15:19:01.263207 27553 raft_consensus.cc:2454] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09629248a1db4e6c9d5436f0fe7d1abb in term 0.
I20260807 15:19:01.263410 27923 raft_consensus.cc:2454] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09629248a1db4e6c9d5436f0fe7d1abb in term 0.
I20260807 15:19:01.263785 27639 leader_election.cc:302] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09629248a1db4e6c9d5436f0fe7d1abb, d74eab887be34ea2a4a971fb770aa11a; no voters:
I20260807 15:19:01.264067 28020 raft_consensus.cc:2781] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:19:01.264158 28020 raft_consensus.cc:482] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:19:01.264209 28020 raft_consensus.cc:3037] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:01.265309 28020 raft_consensus.cc:504] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.265735 28020 leader_election.cc:288] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [CANDIDATE]: Term 1 election: Requested vote from peers d74eab887be34ea2a4a971fb770aa11a (127.31.230.193:38469), 72f2a11167494aa7a6b15d6109d1d6b5 (127.31.230.195:46335)
I20260807 15:19:01.266147 27923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e8e07882704df79d34a35a89ee51e3" candidate_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f2a11167494aa7a6b15d6109d1d6b5"
I20260807 15:19:01.266189 27553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e8e07882704df79d34a35a89ee51e3" candidate_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74eab887be34ea2a4a971fb770aa11a"
I20260807 15:19:01.266288 27923 raft_consensus.cc:3037] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:01.266304 27553 raft_consensus.cc:3037] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:01.267650 27923 raft_consensus.cc:2454] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09629248a1db4e6c9d5436f0fe7d1abb in term 1.
I20260807 15:19:01.267645 27553 raft_consensus.cc:2454] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09629248a1db4e6c9d5436f0fe7d1abb in term 1.
I20260807 15:19:01.267987 27638 leader_election.cc:302] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09629248a1db4e6c9d5436f0fe7d1abb, 72f2a11167494aa7a6b15d6109d1d6b5; no voters:
I20260807 15:19:01.268203 28020 raft_consensus.cc:2781] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:01.268471 28020 raft_consensus.cc:686] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 1 LEADER]: Becoming Leader. State: Replica: 09629248a1db4e6c9d5436f0fe7d1abb, State: Running, Role: LEADER
I20260807 15:19:01.268819 28020 consensus_queue.cc:237] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } }
I20260807 15:19:01.272502 27359 catalog_manager.cc:5495] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb reported cstate change: term changed from 0 to 1, leader changed from <none> to 09629248a1db4e6c9d5436f0fe7d1abb (127.31.230.194). New cstate: current_term: 1 leader_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09629248a1db4e6c9d5436f0fe7d1abb" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 } health_report { overall_health: UNKNOWN } } }
I20260807 15:19:01.342073 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.338382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.31.230.194:43085)
0807 15:19:01.338603 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.338608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.338623 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:19:01.338708 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.338711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.338799 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.338921 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:01.339156 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.339674 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.340515 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.340751 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.340835 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.341128 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:01.341679 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:01.341788 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.341839 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.341891 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 15:19:01.348137 27553 raft_consensus.cc:1261] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 1 FOLLOWER]: Refusing update from remote peer 09629248a1db4e6c9d5436f0fe7d1abb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:01.348136 27923 raft_consensus.cc:1261] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 1 FOLLOWER]: Refusing update from remote peer 09629248a1db4e6c9d5436f0fe7d1abb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:01.348909 28026 consensus_queue.cc:1035] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [LEADER]: Connected to new peer: Peer: permanent_uuid: "72f2a11167494aa7a6b15d6109d1d6b5" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:01.349113 28020 consensus_queue.cc:1035] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74eab887be34ea2a4a971fb770aa11a" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 38469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:01.360579 28033 mvcc.cc:205] Tried to move back new op lower bound from 7315930895754391552 to 7315930895439736832. Current Snapshot: MvccSnapshot[applied={T|T < 7315930895754391552}]
I20260807 15:19:01.366298 27360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:19:01.366460 27360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:19:01.369064 27360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7218d1cd356746a5a1e01b09c9fd3c1b: Sending DeleteTablet for 3 replicas of tablet f8e8e07882704df79d34a35a89ee51e3
I20260807 15:19:01.369932 27530 tablet_service.cc:1514] Processing DeleteTablet for tablet f8e8e07882704df79d34a35a89ee51e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:19:01 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:58804
I20260807 15:19:01.369980 27707 tablet_service.cc:1514] Processing DeleteTablet for tablet f8e8e07882704df79d34a35a89ee51e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:19:01 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:37414
I20260807 15:19:01.370235 27896 tablet_service.cc:1514] Processing DeleteTablet for tablet f8e8e07882704df79d34a35a89ee51e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:19:01 UTC) from {username='kudu', principal='kudu/127.31.230.254@KRBTEST.COM'} at 127.0.0.1:42648
I20260807 15:19:01.370497 28044 tablet_replica.cc:330] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: stopping tablet replica
I20260807 15:19:01.370719 28046 tablet_replica.cc:330] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: stopping tablet replica
I20260807 15:19:01.370923 28044 raft_consensus.cc:2229] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:19:01.371076 28046 raft_consensus.cc:2229] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:19:01.371186 28045 tablet_replica.cc:330] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a: stopping tablet replica
I20260807 15:19:01.371454 28046 raft_consensus.cc:2258] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:19:01.371506 28045 raft_consensus.cc:2229] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:19:01.371682 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 27443
I20260807 15:19:01.371855 28045 raft_consensus.cc:2258] T f8e8e07882704df79d34a35a89ee51e3 P d74eab887be34ea2a4a971fb770aa11a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:19:01.387356 28044 raft_consensus.cc:2258] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:19:01.388459 28046 ts_tablet_manager.cc:1905] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:19:01.391861 28044 ts_tablet_manager.cc:1905] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:19:01.392674 28046 ts_tablet_manager.cc:1918] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:19:01.392899 28046 log.cc:1196] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/f8e8e07882704df79d34a35a89ee51e3
I20260807 15:19:01.393373 28046 ts_tablet_manager.cc:1939] T f8e8e07882704df79d34a35a89ee51e3 P 72f2a11167494aa7a6b15d6109d1d6b5: Deleting consensus metadata
I20260807 15:19:01.394727 28044 ts_tablet_manager.cc:1918] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:19:01.394865 28044 log.cc:1196] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/f8e8e07882704df79d34a35a89ee51e3
I20260807 15:19:01.395218 28044 ts_tablet_manager.cc:1939] T f8e8e07882704df79d34a35a89ee51e3 P 09629248a1db4e6c9d5436f0fe7d1abb: Deleting consensus metadata
I20260807 15:19:01.395174 27340 catalog_manager.cc:4841] TS 72f2a11167494aa7a6b15d6109d1d6b5 (127.31.230.195:46335): tablet f8e8e07882704df79d34a35a89ee51e3 (table test-table [id=4c2b5f1dc25c4754ab710e9adf76daf5]) successfully deleted
I20260807 15:19:01.396476 27343 catalog_manager.cc:4841] TS 09629248a1db4e6c9d5436f0fe7d1abb (127.31.230.194:43085): tablet f8e8e07882704df79d34a35a89ee51e3 (table test-table [id=4c2b5f1dc25c4754ab710e9adf76daf5]) successfully deleted
W20260807 15:19:01.397522 27341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:38469 (error 108)
I20260807 15:19:01.398559 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 27620
I20260807 15:19:01.398749 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.398409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:38469 (local address 127.0.0.1:58812)
0807 15:19:01.398593 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.398665 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:38469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260807 15:19:01.405725 27341 catalog_manager.cc:4584] TS d74eab887be34ea2a4a971fb770aa11a (127.31.230.193:38469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8e8e07882704df79d34a35a89ee51e3: Network error: Client connection negotiation failed: client connection to 127.31.230.193:38469: connect: Connection refused (error 111)
I20260807 15:19:01.414824 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 27807
I20260807 15:19:01.429086 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 27319
2026-08-07T15:19:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2809 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T15:19:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:01Z Disabled control of system clock
I20260807 15:19:01.470031 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39829
--webserver_interface=127.31.230.254
--webserver_port=0
--builtin_ntp_servers=127.31.230.212:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:19:01.587589 28051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:01.587875 28051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:01.587913 28051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:01.591980 28051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:01.592044 28051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:01.592059 28051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:01.592077 28051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:01.592092 28051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:01.596987 28051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:39829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:01.598136 28051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:01.599519 28051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:01.608426 28064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:01.608397 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:01.608438 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:01.608572 28051 server_base.cc:1034] running on GCE node
I20260807 15:19:01.609292 28051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:01.610532 28051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:01.611750 28051 hybrid_clock.cc:648] HybridClock initialized: now 1786115941611722 us; error 46 us; skew 500 ppm
I20260807 15:19:01.620599 28051 webserver.cc:460] Webserver started at http://127.31.230.254:40903/ using document root <none> and password file <none>
I20260807 15:19:01.621296 28051 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:01.621407 28051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:01.621726 28051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:01.623968 28051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "4ca14114c88c48b5bc61ae8570b30604"
format_stamp: "Formatted at 2026-08-07 15:19:01 on dist-test-slave-gkw7"
server_key: "268bac9807da6121af62cb4549eb63b0"
server_key_iv: "9e8e7f6372936161969996fb521d23c0"
server_key_version: "encryptionkey@0"
I20260807 15:19:01.624588 28051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "4ca14114c88c48b5bc61ae8570b30604"
format_stamp: "Formatted at 2026-08-07 15:19:01 on dist-test-slave-gkw7"
server_key: "268bac9807da6121af62cb4549eb63b0"
server_key_iv: "9e8e7f6372936161969996fb521d23c0"
server_key_version: "encryptionkey@0"
I20260807 15:19:01.629035 28051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:19:01.631735 28076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:01.633140 28051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:19:01.633323 28051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "4ca14114c88c48b5bc61ae8570b30604"
format_stamp: "Formatted at 2026-08-07 15:19:01 on dist-test-slave-gkw7"
server_key: "268bac9807da6121af62cb4549eb63b0"
server_key_iv: "9e8e7f6372936161969996fb521d23c0"
server_key_version: "encryptionkey@0"
I20260807 15:19:01.633453 28051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:01.661998 28051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:01.662902 28051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:01.663074 28051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:01.670293 28051 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:39829
I20260807 15:19:01.670308 28150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:39829 every 8 connection(s)
I20260807 15:19:01.671288 28051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:01.674669 28153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:01.676985 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 28051
I20260807 15:19:01.677187 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:01.677484 32667 external_mini_cluster.cc:1428] Setting key 0ca186b22df04b0b8548e16f63c1499a
I20260807 15:19:01.681041 28153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604: Bootstrap starting.
I20260807 15:19:01.683722 28153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:01.684579 28153 log.cc:824] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:01.686875 28153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604: No bootstrap required, opened a new log
I20260807 15:19:01.687465 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.679069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51990 (local address 127.31.230.254:39829)
0807 15:19:01.679780 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.679796 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.679833 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:19:01.680487 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.680504 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.680919 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.681291 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:01.681736 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.682740 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.683987 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.684463 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.684591 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.685484 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:01.685489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:01.685507 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:01.685968 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:01.686027 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:01.686048 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:01.686182 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.686335 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.686602 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":151,"threads_started":1}
I20260807 15:19:01.691531 28153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca14114c88c48b5bc61ae8570b30604" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39829 } }
I20260807 15:19:01.691768 28153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:01.691844 28153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca14114c88c48b5bc61ae8570b30604, State: Initialized, Role: FOLLOWER
I20260807 15:19:01.692310 28153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca14114c88c48b5bc61ae8570b30604" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39829 } }
I20260807 15:19:01.692461 28153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:01.692525 28153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:01.692646 28153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:01.693876 28153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca14114c88c48b5bc61ae8570b30604" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39829 } }
I20260807 15:19:01.694260 28153 leader_election.cc:302] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ca14114c88c48b5bc61ae8570b30604; no voters:
I20260807 15:19:01.694656 28153 leader_election.cc:288] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:01.694783 28163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:01.695096 28163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca14114c88c48b5bc61ae8570b30604, State: Running, Role: LEADER
I20260807 15:19:01.695470 28163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca14114c88c48b5bc61ae8570b30604" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39829 } }
I20260807 15:19:01.696035 28153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:01.697384 28165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ca14114c88c48b5bc61ae8570b30604. Latest consensus state: current_term: 1 leader_uuid: "4ca14114c88c48b5bc61ae8570b30604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca14114c88c48b5bc61ae8570b30604" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39829 } } }
I20260807 15:19:01.697513 28165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:01.697357 28164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ca14114c88c48b5bc61ae8570b30604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ca14114c88c48b5bc61ae8570b30604" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39829 } } }
I20260807 15:19:01.697788 28164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:01.698601 28173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:01.701385 28173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:01.707345 28173 catalog_manager.cc:1347] Generated new cluster ID: f842d726874b4154811923e4786392b2
I20260807 15:19:01.707419 28173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:01.730223 28173 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:01.731560 28173 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:01.742504 28173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604: Generated new TSK 0
I20260807 15:19:01.743395 28173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:01.750910 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.193:0
--local_ip_for_outbound_sockets=127.31.230.193
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39829
--builtin_ntp_servers=127.31.230.212:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:19:01.872648 28188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:01.872937 28188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:01.872978 28188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:01.877168 28188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:01.877259 28188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:01.877319 28188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.193
I20260807 15:19:01.882884 28188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.230.193
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.193
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:01.883999 28188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:01.885408 28188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:01.895654 28195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:01.895745 28194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:01.895869 28188 server_base.cc:1034] running on GCE node
W20260807 15:19:01.895654 28197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:01.896540 28188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:01.897392 28188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:01.898607 28188 hybrid_clock.cc:648] HybridClock initialized: now 1786115941898562 us; error 58 us; skew 500 ppm
I20260807 15:19:01.900774 28188 webserver.cc:460] Webserver started at http://127.31.230.193:38665/ using document root <none> and password file <none>
I20260807 15:19:01.901410 28188 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:01.901511 28188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:01.901778 28188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:01.904007 28188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data/instance:
uuid: "2c8b30850e664e6a84d687343ba6e5c0"
format_stamp: "Formatted at 2026-08-07 15:19:01 on dist-test-slave-gkw7"
server_key: "83fc1dabc24b777fb3fe3eb6740b8b44"
server_key_iv: "080cd573d9b62a78ac97d9772cd67001"
server_key_version: "encryptionkey@0"
I20260807 15:19:01.904635 28188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance:
uuid: "2c8b30850e664e6a84d687343ba6e5c0"
format_stamp: "Formatted at 2026-08-07 15:19:01 on dist-test-slave-gkw7"
server_key: "83fc1dabc24b777fb3fe3eb6740b8b44"
server_key_iv: "080cd573d9b62a78ac97d9772cd67001"
server_key_version: "encryptionkey@0"
I20260807 15:19:01.908715 28188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 15:19:01.911414 28205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:01.912657 28188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:01.912802 28188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal
uuid: "2c8b30850e664e6a84d687343ba6e5c0"
format_stamp: "Formatted at 2026-08-07 15:19:01 on dist-test-slave-gkw7"
server_key: "83fc1dabc24b777fb3fe3eb6740b8b44"
server_key_iv: "080cd573d9b62a78ac97d9772cd67001"
server_key_version: "encryptionkey@0"
I20260807 15:19:01.912927 28188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:01.929440 28188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:01.930218 28188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:01.930440 28188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:01.931087 28188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:19:01.932081 28188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:19:01.932156 28188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:01.932235 28188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:19:01.932279 28188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:01.941685 28188 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.193:34629
I20260807 15:19:01.941735 28364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.193:34629 every 8 connection(s)
I20260807 15:19:01.942734 28188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/data/info.pb
I20260807 15:19:01.949055 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 28188
I20260807 15:19:01.949204 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-0/wal/instance
I20260807 15:19:01.949502 32667 external_mini_cluster.cc:1428] Setting key a9d63781e8615d5599d4149c5e21a16e
I20260807 15:19:01.951897 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.194:0
--local_ip_for_outbound_sockets=127.31.230.194
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39829
--builtin_ntp_servers=127.31.230.212:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:19:01.953181 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.944698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:52341 (local address 127.31.230.254:39829)
0807 15:19:01.944853 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:01.944859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:01.946091 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 15:19:01.947555 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:01.947558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:01.947618 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:01.947724 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:01.948653 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.949503 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.950764 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.951087 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.951187 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.952642 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:01.952646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:01.952648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:01.952659 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:01.952682 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:01.952690 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:01.952755 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:01.952985 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:19:01.953046 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:19:01.954356 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:01.945156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39829 (local address 127.31.230.193:52341)
0807 15:19:01.945930 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:01.945974 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:01.947324 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:01.947918 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:01.947925 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:01.948012 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:01.948506 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.948519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.949694 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.949697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.950627 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:01.950638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:01.951289 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:01.951294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:01.951504 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:01.952305 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:19:01.952309 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:01.952369 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:19:01.952461 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:01.952787 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:01.952800 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:01.952810 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 15:19:01.952960 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 15:19:01.953298 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":111,"threads_started":1}
I20260807 15:19:01.955788 28365 heartbeater.cc:347] Connected to a master server at 127.31.230.254:39829
I20260807 15:19:01.956050 28365 heartbeater.cc:464] Registering TS with master...
I20260807 15:19:01.956602 28365 heartbeater.cc:511] Master 127.31.230.254:39829 requested a full tablet report, sending...
I20260807 15:19:01.958437 28101 ts_manager.cc:194] Registered new tserver with Master: 2c8b30850e664e6a84d687343ba6e5c0 (127.31.230.193:34629)
I20260807 15:19:01.960309 28101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.193:52341
W20260807 15:19:02.073465 28368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:02.073804 28368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:02.073846 28368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:02.077963 28368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:02.078034 28368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:02.078092 28368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.194
I20260807 15:19:02.082924 28368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.230.194
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.194
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:02.084060 28368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:02.085459 28368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:02.095788 28375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:02.095769 28374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:02.095788 28377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:02.096104 28368 server_base.cc:1034] running on GCE node
I20260807 15:19:02.096716 28368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:02.097573 28368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:02.098753 28368 hybrid_clock.cc:648] HybridClock initialized: now 1786115942098728 us; error 35 us; skew 500 ppm
I20260807 15:19:02.100679 28368 webserver.cc:460] Webserver started at http://127.31.230.194:45181/ using document root <none> and password file <none>
I20260807 15:19:02.101307 28368 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:02.101374 28368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:02.101622 28368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:02.103780 28368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data/instance:
uuid: "9866a2689f43432288f1aa1e2cd71d1b"
format_stamp: "Formatted at 2026-08-07 15:19:02 on dist-test-slave-gkw7"
server_key: "90a05fa1a1298eb6bb74a9135721ff7a"
server_key_iv: "eff1686a7c29f077859b54cdd50faa6f"
server_key_version: "encryptionkey@0"
I20260807 15:19:02.104334 28368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance:
uuid: "9866a2689f43432288f1aa1e2cd71d1b"
format_stamp: "Formatted at 2026-08-07 15:19:02 on dist-test-slave-gkw7"
server_key: "90a05fa1a1298eb6bb74a9135721ff7a"
server_key_iv: "eff1686a7c29f077859b54cdd50faa6f"
server_key_version: "encryptionkey@0"
I20260807 15:19:02.108201 28368 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260807 15:19:02.110883 28386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:02.112126 28368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:02.112284 28368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal
uuid: "9866a2689f43432288f1aa1e2cd71d1b"
format_stamp: "Formatted at 2026-08-07 15:19:02 on dist-test-slave-gkw7"
server_key: "90a05fa1a1298eb6bb74a9135721ff7a"
server_key_iv: "eff1686a7c29f077859b54cdd50faa6f"
server_key_version: "encryptionkey@0"
I20260807 15:19:02.112394 28368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:02.131845 28368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:02.132602 28368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:02.132844 28368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:02.133491 28368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:19:02.134637 28368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:19:02.134687 28368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:02.134728 28368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:19:02.134747 28368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:02.144258 28368 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.194:35459
I20260807 15:19:02.144279 28532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.194:35459 every 8 connection(s)
I20260807 15:19:02.145208 28368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/data/info.pb
I20260807 15:19:02.150944 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 28368
I20260807 15:19:02.151096 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal/instance
I20260807 15:19:02.151436 32667 external_mini_cluster.cc:1428] Setting key ba8a758b8b03a49c915e83397d0bd550
I20260807 15:19:02.153776 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.230.195:0
--local_ip_for_outbound_sockets=127.31.230.195
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39829
--builtin_ntp_servers=127.31.230.212:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:19:02.155746 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.147194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.194:55611 (local address 127.31.230.254:39829)
0807 15:19:02.147346 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.147351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.148488 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 15:19:02.149946 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.149949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.150013 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.150091 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:02.151206 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.151972 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.153603 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.153854 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.153980 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.155234 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:02.155241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:02.155243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:02.155255 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:02.155274 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:02.155282 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:02.155365 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.155535 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.155598 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:19:02.156633 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.147637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39829 (local address 127.31.230.194:55611)
0807 15:19:02.148331 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.148375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.149740 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.150269 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.150276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.150362 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:02.151002 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.151026 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.152104 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.152108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.153412 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.153428 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.154132 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.154134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.154290 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.154950 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:19:02.154951 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:02.154992 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:19:02.155057 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:02.155367 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:02.155379 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:02.155388 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.155530 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.155843 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260807 15:19:02.157931 28533 heartbeater.cc:347] Connected to a master server at 127.31.230.254:39829
I20260807 15:19:02.158183 28533 heartbeater.cc:464] Registering TS with master...
I20260807 15:19:02.158730 28533 heartbeater.cc:511] Master 127.31.230.254:39829 requested a full tablet report, sending...
I20260807 15:19:02.159912 28101 ts_manager.cc:194] Registered new tserver with Master: 9866a2689f43432288f1aa1e2cd71d1b (127.31.230.194:35459)
I20260807 15:19:02.160840 28101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.194:55611
W20260807 15:19:02.272087 28536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:02.272408 28536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:02.272454 28536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:02.276629 28536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:02.276701 28536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:02.276764 28536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.230.195
I20260807 15:19:02.281751 28536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.230.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.230.195
--webserver_port=0
--tserver_master_addrs=127.31.230.254:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.230.195
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:02.282985 28536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:02.284479 28536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:02.294736 28543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:02.294759 28544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:02.294765 28546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:02.295071 28536 server_base.cc:1034] running on GCE node
I20260807 15:19:02.295477 28536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:02.296186 28536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:02.297397 28536 hybrid_clock.cc:648] HybridClock initialized: now 1786115942297355 us; error 60 us; skew 500 ppm
I20260807 15:19:02.299474 28536 webserver.cc:460] Webserver started at http://127.31.230.195:40435/ using document root <none> and password file <none>
I20260807 15:19:02.300092 28536 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:02.300191 28536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:02.300422 28536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:02.302664 28536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data/instance:
uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a"
format_stamp: "Formatted at 2026-08-07 15:19:02 on dist-test-slave-gkw7"
server_key: "e2cb91ae1ffa0b9a0d057d06ca916d1e"
server_key_iv: "1f81b7cfb162653b41f238950e5c3077"
server_key_version: "encryptionkey@0"
I20260807 15:19:02.303262 28536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance:
uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a"
format_stamp: "Formatted at 2026-08-07 15:19:02 on dist-test-slave-gkw7"
server_key: "e2cb91ae1ffa0b9a0d057d06ca916d1e"
server_key_iv: "1f81b7cfb162653b41f238950e5c3077"
server_key_version: "encryptionkey@0"
I20260807 15:19:02.307156 28536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:19:02.309855 28553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:02.311172 28536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:19:02.311347 28536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal
uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a"
format_stamp: "Formatted at 2026-08-07 15:19:02 on dist-test-slave-gkw7"
server_key: "e2cb91ae1ffa0b9a0d057d06ca916d1e"
server_key_iv: "1f81b7cfb162653b41f238950e5c3077"
server_key_version: "encryptionkey@0"
I20260807 15:19:02.311477 28536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:02.336066 28536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:02.336974 28536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:02.337215 28536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:02.337936 28536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:19:02.338967 28536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:19:02.339043 28536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:02.339119 28536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:19:02.339174 28536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:02.349346 28536 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.195:37791
I20260807 15:19:02.349370 28700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.195:37791 every 8 connection(s)
I20260807 15:19:02.350371 28536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/data/info.pb
I20260807 15:19:02.352411 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 28536
I20260807 15:19:02.352536 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal/instance
I20260807 15:19:02.352833 32667 external_mini_cluster.cc:1428] Setting key c8e1bb8435d021b0272f572ce0bb4734
I20260807 15:19:02.360630 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.352452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.195:40807 (local address 127.31.230.254:39829)
0807 15:19:02.352623 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.352629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.354101 (+ 1472us) server_negotiation.cc:407] Connection header received
0807 15:19:02.355826 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.355829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.355876 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.355946 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:02.356896 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.357434 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.358701 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.358964 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.359076 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.360156 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:02.360160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:02.360162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:02.360174 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:02.360194 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:02.360202 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:02.360265 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.360431 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.360489 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:19:02.361471 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.352981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:39829 (local address 127.31.230.195:40807)
0807 15:19:02.353912 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.353971 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.355602 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.356151 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.356158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.356247 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:02.356747 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.356759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.357635 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.357639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.358551 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.358561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.359111 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.359114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.359289 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.359898 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:19:02.359899 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:02.359941 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:19:02.360006 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:02.360280 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:02.360287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:02.360291 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.360389 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.360617 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":161,"threads_started":1}
I20260807 15:19:02.362808 28701 heartbeater.cc:347] Connected to a master server at 127.31.230.254:39829
I20260807 15:19:02.363035 28701 heartbeater.cc:464] Registering TS with master...
I20260807 15:19:02.363556 28701 heartbeater.cc:511] Master 127.31.230.254:39829 requested a full tablet report, sending...
I20260807 15:19:02.364722 28101 ts_manager.cc:194] Registered new tserver with Master: 5dd7f22bc0dd4029b9031b8322c7bb2a (127.31.230.195:37791)
I20260807 15:19:02.365463 28101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.230.195:40807
I20260807 15:19:02.367350 32667 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:19:02.373291 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.369641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.31.230.254:39829)
0807 15:19:02.369803 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.369808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.370018 (+ 210us) server_negotiation.cc:407] Connection header received
0807 15:19:02.370110 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.370112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.370152 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.370218 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:02.370432 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.370927 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.371732 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.371950 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.372034 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.372829 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:02.372832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:02.372834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:02.372846 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:02.372865 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:02.372873 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:02.372943 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.373013 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.373064 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:19:02.376529 28101 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:19:02.379765 28101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:19:02.394941 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.388848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.31.230.194:35459)
0807 15:19:02.389601 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.389622 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.389648 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:19:02.389730 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.389736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.389957 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.390296 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:02.390624 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.391790 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.393587 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.394183 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.394321 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.394515 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.394641 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.394747 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":114,"threads_started":1}
I20260807 15:19:02.394968 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.388681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35459 (local address 127.0.0.1:47472)
0807 15:19:02.389158 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.389206 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.389337 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.390303 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.390313 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.390347 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:02.390515 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.390522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.391921 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.391925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.393427 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.393438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.394320 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.394324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.394441 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.394479 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.394611 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.394823 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":65,"spinlock_wait_cycles":384,"thread_start_us":99,"threads_started":1}
I20260807 15:19:02.395349 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.388828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37791 (local address 127.0.0.1:38546)
0807 15:19:02.390245 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.390260 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.390357 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.390873 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.390876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.390893 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:02.390976 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.390983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.392321 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.392324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.393602 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.393610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.394850 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.394854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.395016 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.395043 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.395127 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.395212 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"mutex_wait_us":65,"spinlock_wait_cycles":6272,"thread_start_us":125,"threads_started":1}
I20260807 15:19:02.395964 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.389430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:34629 (local address 127.0.0.1:58436)
0807 15:19:02.390371 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.390386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.390481 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.391046 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.391050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.391069 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:02.391137 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.391141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.392192 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.392198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.393526 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.393566 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.394850 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.394854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.395609 (+ 755us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.395644 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.395766 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.395830 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":385,"thread_start_us":119,"threads_started":1}
I20260807 15:19:02.395973 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.389731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38546 (local address 127.31.230.195:37791)
0807 15:19:02.390103 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.390114 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.390456 (+ 342us) server_negotiation.cc:407] Connection header received
0807 15:19:02.390543 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.390551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.390743 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.390894 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:02.391125 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.392157 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.393750 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.394493 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.394617 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.394781 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.395686 (+ 905us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.395803 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":95,"threads_started":1}
I20260807 15:19:02.396288 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.389651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.31.230.193:34629)
0807 15:19:02.390097 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.390111 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.390478 (+ 367us) server_negotiation.cc:407] Connection header received
0807 15:19:02.390590 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.390595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.390794 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.390945 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:02.391255 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.392065 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.393706 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.394493 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.394617 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.394781 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.395837 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.395953 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":129,"threads_started":1}
I20260807 15:19:02.398950 28266 tablet_service.cc:1467] Processing CreateTablet for tablet 62713789b68345cb88ff30cb04fdce7a (DEFAULT_TABLE table=test-table [id=73fd980b79d54e5fa33cc07d80b65093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:19:02.398978 28616 tablet_service.cc:1467] Processing CreateTablet for tablet 62713789b68345cb88ff30cb04fdce7a (DEFAULT_TABLE table=test-table [id=73fd980b79d54e5fa33cc07d80b65093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:19:02.398978 28451 tablet_service.cc:1467] Processing CreateTablet for tablet 62713789b68345cb88ff30cb04fdce7a (DEFAULT_TABLE table=test-table [id=73fd980b79d54e5fa33cc07d80b65093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:19:02.400084 28616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62713789b68345cb88ff30cb04fdce7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:02.400085 28266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62713789b68345cb88ff30cb04fdce7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:02.400341 28451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62713789b68345cb88ff30cb04fdce7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:02.406595 28726 tablet_bootstrap.cc:492] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: Bootstrap starting.
I20260807 15:19:02.408151 28727 tablet_bootstrap.cc:492] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Bootstrap starting.
I20260807 15:19:02.408586 28728 tablet_bootstrap.cc:492] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Bootstrap starting.
I20260807 15:19:02.409121 28726 tablet_bootstrap.cc:654] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:02.410123 28727 tablet_bootstrap.cc:654] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:02.410099 28726 log.cc:824] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:02.410708 28728 tablet_bootstrap.cc:654] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:02.411285 28727 log.cc:824] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:02.411511 28728 log.cc:824] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:02.412868 28726 tablet_bootstrap.cc:492] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: No bootstrap required, opened a new log
I20260807 15:19:02.413079 28726 ts_tablet_manager.cc:1397] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:19:02.413375 28727 tablet_bootstrap.cc:492] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: No bootstrap required, opened a new log
I20260807 15:19:02.413607 28727 ts_tablet_manager.cc:1397] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:19:02.414642 28728 tablet_bootstrap.cc:492] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: No bootstrap required, opened a new log
I20260807 15:19:02.414844 28728 ts_tablet_manager.cc:1397] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:19:02.417969 28727 raft_consensus.cc:348] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.417969 28726 raft_consensus.cc:348] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.418265 28727 raft_consensus.cc:374] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:02.418321 28727 raft_consensus.cc:729] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9866a2689f43432288f1aa1e2cd71d1b, State: Initialized, Role: FOLLOWER
I20260807 15:19:02.418265 28726 raft_consensus.cc:374] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:02.418401 28726 raft_consensus.cc:729] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8b30850e664e6a84d687343ba6e5c0, State: Initialized, Role: FOLLOWER
I20260807 15:19:02.418856 28726 consensus_queue.cc:260] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.418856 28727 consensus_queue.cc:260] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.419286 28728 raft_consensus.cc:348] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.419541 28728 raft_consensus.cc:374] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:02.419586 28533 heartbeater.cc:503] Master 127.31.230.254:39829 was elected leader, sending a full tablet report...
I20260807 15:19:02.419605 28728 raft_consensus.cc:729] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd7f22bc0dd4029b9031b8322c7bb2a, State: Initialized, Role: FOLLOWER
I20260807 15:19:02.419925 28726 ts_tablet_manager.cc:1428] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:19:02.419925 28727 ts_tablet_manager.cc:1428] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:19:02.420003 28728 consensus_queue.cc:260] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.420275 28365 heartbeater.cc:503] Master 127.31.230.254:39829 was elected leader, sending a full tablet report...
I20260807 15:19:02.420689 28701 heartbeater.cc:503] Master 127.31.230.254:39829 was elected leader, sending a full tablet report...
I20260807 15:19:02.421012 28728 ts_tablet_manager.cc:1428] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 15:19:02.444018 28366 tablet.cc:2369] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:19:02.484170 28737 raft_consensus.cc:482] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:19:02.484387 28737 raft_consensus.cc:504] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.485730 28737 leader_election.cc:288] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9866a2689f43432288f1aa1e2cd71d1b (127.31.230.194:35459), 5dd7f22bc0dd4029b9031b8322c7bb2a (127.31.230.195:37791)
I20260807 15:19:02.489849 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.486066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:43935 (local address 127.31.230.194:35459)
0807 15:19:02.486215 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.486220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.486380 (+ 160us) server_negotiation.cc:407] Connection header received
0807 15:19:02.486628 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.486632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.486697 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.486758 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:02.487165 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.487696 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.488828 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.489236 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.489359 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.489415 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.489625 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.489705 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:19:02.489858 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.486448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.230.193:52251 (local address 127.31.230.195:37791)
0807 15:19:02.486618 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.486623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.486643 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:19:02.486691 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.486694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.486731 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.486838 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:19:02.487202 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.487688 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.488874 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.489257 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.489394 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.489445 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.489681 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.489753 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 15:19:02.489863 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.485995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.195:37791 (local address 127.31.230.193:52251)
0807 15:19:02.486450 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.486464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.486569 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.486876 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.486879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.486902 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:02.487032 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.487043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.487807 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.487813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.488707 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.488718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.489407 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.489410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.489507 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.489524 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.489647 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.489710 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":83,"thread_start_us":147,"threads_started":1}
I20260807 15:19:02.489879 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.485897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.194:35459 (local address 127.31.230.193:43935)
0807 15:19:02.486289 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.486305 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:02.486475 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:02.486806 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:02.486811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:02.486832 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:19:02.487004 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.487011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.487846 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.487848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.488697 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:02.488706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.489370 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.489374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:02.489475 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.489504 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:19:02.489609 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 15:19:02.489695 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":4,"thread_start_us":125,"threads_started":1}
I20260807 15:19:02.490533 28478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62713789b68345cb88ff30cb04fdce7a" candidate_uuid: "2c8b30850e664e6a84d687343ba6e5c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9866a2689f43432288f1aa1e2cd71d1b" is_pre_election: true
I20260807 15:19:02.490533 28644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62713789b68345cb88ff30cb04fdce7a" candidate_uuid: "2c8b30850e664e6a84d687343ba6e5c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" is_pre_election: true
I20260807 15:19:02.490876 28644 raft_consensus.cc:2454] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8b30850e664e6a84d687343ba6e5c0 in term 0.
I20260807 15:19:02.490876 28478 raft_consensus.cc:2454] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8b30850e664e6a84d687343ba6e5c0 in term 0.
I20260807 15:19:02.491425 28208 leader_election.cc:302] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8b30850e664e6a84d687343ba6e5c0, 9866a2689f43432288f1aa1e2cd71d1b; no voters:
I20260807 15:19:02.491726 28737 raft_consensus.cc:2781] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:19:02.491834 28737 raft_consensus.cc:482] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:19:02.491895 28737 raft_consensus.cc:3037] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:02.492993 28737 raft_consensus.cc:504] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.493386 28737 leader_election.cc:288] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [CANDIDATE]: Term 1 election: Requested vote from peers 9866a2689f43432288f1aa1e2cd71d1b (127.31.230.194:35459), 5dd7f22bc0dd4029b9031b8322c7bb2a (127.31.230.195:37791)
I20260807 15:19:02.493779 28644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62713789b68345cb88ff30cb04fdce7a" candidate_uuid: "2c8b30850e664e6a84d687343ba6e5c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a"
I20260807 15:19:02.493917 28644 raft_consensus.cc:3037] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:02.494993 28644 raft_consensus.cc:2454] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8b30850e664e6a84d687343ba6e5c0 in term 1.
I20260807 15:19:02.495280 28478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62713789b68345cb88ff30cb04fdce7a" candidate_uuid: "2c8b30850e664e6a84d687343ba6e5c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9866a2689f43432288f1aa1e2cd71d1b"
I20260807 15:19:02.495359 28207 leader_election.cc:302] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8b30850e664e6a84d687343ba6e5c0, 5dd7f22bc0dd4029b9031b8322c7bb2a; no voters:
I20260807 15:19:02.495432 28478 raft_consensus.cc:3037] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:02.495553 28737 raft_consensus.cc:2781] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:02.495817 28737 raft_consensus.cc:686] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8b30850e664e6a84d687343ba6e5c0, State: Running, Role: LEADER
I20260807 15:19:02.496248 28737 consensus_queue.cc:237] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } }
I20260807 15:19:02.496764 28478 raft_consensus.cc:2454] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8b30850e664e6a84d687343ba6e5c0 in term 1.
I20260807 15:19:02.499964 28101 catalog_manager.cc:5495] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8b30850e664e6a84d687343ba6e5c0 (127.31.230.193). New cstate: current_term: 1 leader_uuid: "2c8b30850e664e6a84d687343ba6e5c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8b30850e664e6a84d687343ba6e5c0" member_type: VOTER last_known_addr { host: "127.31.230.193" port: 34629 } health_report { overall_health: HEALTHY } } }
I20260807 15:19:02.519104 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.515185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.31.230.193:34629)
0807 15:19:02.515342 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:02.515347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:02.515363 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:19:02.515530 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:02.515532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:02.515602 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:02.515718 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:02.516015 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.516552 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.517577 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:02.517808 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:02.517944 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:02.518272 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:02.518712 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:02.518855 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:02.518896 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 15:19:02.518953 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:19:02.525442 28478 raft_consensus.cc:1261] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 1 FOLLOWER]: Refusing update from remote peer 2c8b30850e664e6a84d687343ba6e5c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:02.525461 28644 raft_consensus.cc:1261] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 1 FOLLOWER]: Refusing update from remote peer 2c8b30850e664e6a84d687343ba6e5c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:02.526240 28742 consensus_queue.cc:1035] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9866a2689f43432288f1aa1e2cd71d1b" member_type: VOTER last_known_addr { host: "127.31.230.194" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:02.526466 28737 consensus_queue.cc:1035] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd7f22bc0dd4029b9031b8322c7bb2a" member_type: VOTER last_known_addr { host: "127.31.230.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:02.535068 28749 mvcc.cc:205] Tried to move back new op lower bound from 7315930900576165888 to 7315930900468137984. Current Snapshot: MvccSnapshot[applied={T|T < 7315930900576165888}]
I20260807 15:19:02.535669 28748 mvcc.cc:205] Tried to move back new op lower bound from 7315930900576165888 to 7315930900468137984. Current Snapshot: MvccSnapshot[applied={T|T < 7315930900576165888}]
I20260807 15:19:02.543974 28098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:19:02.544220 28098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:19:02.546813 28098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ca14114c88c48b5bc61ae8570b30604: Sending DeleteTablet for 3 replicas of tablet 62713789b68345cb88ff30cb04fdce7a
I20260807 15:19:02.547648 28451 tablet_service.cc:1514] Processing DeleteTablet for tablet 62713789b68345cb88ff30cb04fdce7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:19:02 UTC) from {username='slave'} at 127.0.0.1:47472
I20260807 15:19:02.547724 28266 tablet_service.cc:1514] Processing DeleteTablet for tablet 62713789b68345cb88ff30cb04fdce7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:19:02 UTC) from {username='slave'} at 127.0.0.1:58436
I20260807 15:19:02.547761 28616 tablet_service.cc:1514] Processing DeleteTablet for tablet 62713789b68345cb88ff30cb04fdce7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:19:02 UTC) from {username='slave'} at 127.0.0.1:38546
I20260807 15:19:02.548235 28764 tablet_replica.cc:330] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: stopping tablet replica
I20260807 15:19:02.548378 28768 tablet_replica.cc:330] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: stopping tablet replica
I20260807 15:19:02.548595 28768 raft_consensus.cc:2229] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:19:02.548579 28764 raft_consensus.cc:2229] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:19:02.548916 28768 raft_consensus.cc:2258] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:19:02.548916 28764 raft_consensus.cc:2258] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:19:02.549093 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 28188
I20260807 15:19:02.549309 28766 tablet_replica.cc:330] T 62713789b68345cb88ff30cb04fdce7a P 2c8b30850e664e6a84d687343ba6e5c0: stopping tablet replica
I20260807 15:19:02.552728 28768 ts_tablet_manager.cc:1905] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:19:02.555094 28768 ts_tablet_manager.cc:1918] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:19:02.555177 28768 log.cc:1196] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-2/wal/wals/62713789b68345cb88ff30cb04fdce7a
I20260807 15:19:02.555516 28768 ts_tablet_manager.cc:1939] T 62713789b68345cb88ff30cb04fdce7a P 5dd7f22bc0dd4029b9031b8322c7bb2a: Deleting consensus metadata
I20260807 15:19:02.556388 28764 ts_tablet_manager.cc:1905] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:19:02.556879 28079 catalog_manager.cc:4841] TS 5dd7f22bc0dd4029b9031b8322c7bb2a (127.31.230.195:37791): tablet 62713789b68345cb88ff30cb04fdce7a (table test-table [id=73fd980b79d54e5fa33cc07d80b65093]) successfully deleted
I20260807 15:19:02.558800 28764 ts_tablet_manager.cc:1918] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:19:02.558923 28764 log.cc:1196] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Deleting WAL directory at /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115680745020-32667-0/minicluster-data/ts-1/wal/wals/62713789b68345cb88ff30cb04fdce7a
I20260807 15:19:02.559257 28764 ts_tablet_manager.cc:1939] T 62713789b68345cb88ff30cb04fdce7a P 9866a2689f43432288f1aa1e2cd71d1b: Deleting consensus metadata
W20260807 15:19:02.560340 28079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.230.193:34629 (error 108)
I20260807 15:19:02.560678 28081 catalog_manager.cc:4841] TS 9866a2689f43432288f1aa1e2cd71d1b (127.31.230.194:35459): tablet 62713789b68345cb88ff30cb04fdce7a (table test-table [id=73fd980b79d54e5fa33cc07d80b65093]) successfully deleted
I20260807 15:19:02.561283 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 28368
I20260807 15:19:02.561483 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:02.561140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.193:34629 (local address 127.0.0.1:58456)
0807 15:19:02.561277 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:02.561381 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.193:34629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:19:02.563900 28079 catalog_manager.cc:4584] TS 2c8b30850e664e6a84d687343ba6e5c0 (127.31.230.193:34629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62713789b68345cb88ff30cb04fdce7a: Network error: Client connection negotiation failed: client connection to 127.31.230.193:34629: connect: Connection refused (error 111)
I20260807 15:19:02.571286 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 28536
I20260807 15:19:02.592500 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 28051
2026-08-07T15:19:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1161 ms)
[----------] 6 tests from EncryptionPolicyTest (11912 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:02 dist-test-slave-gkw7 krb5kdc[28772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:02 dist-test-slave-gkw7 krb5kdc[28772](info): set up 2 sockets
Aug 07 15:19:02 dist-test-slave-gkw7 krb5kdc[28772](info): commencing operation
krb5kdc: starting...
W20260807 15:19:03.818310 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:03 dist-test-slave-gkw7 krb5kdc[28772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:03.954972 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34411
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:04.071419 28794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:04.071774 28794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:04.071883 28794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:04.075994 28794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:04.076109 28794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:04.076148 28794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:04.076186 28794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:04.076220 28794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:04.081758 28794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34411
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:04.082968 28794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:04.084121 28794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:04.093138 28805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:04.093150 28803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:04.093154 28801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:04.093488 28794 server_base.cc:1034] running on GCE node
I20260807 15:19:04.094277 28794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:04.095363 28794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:04.096529 28794 hybrid_clock.cc:648] HybridClock initialized: now 1786115944096507 us; error 34 us; skew 500 ppm
Aug 07 15:19:04 dist-test-slave-gkw7 krb5kdc[28772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115944, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:04.099462 28794 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:04.100417 28794 webserver.cc:460] Webserver started at http://127.0.0.1:45809/ using document root <none> and password file <none>
I20260807 15:19:04.100973 28794 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:04.101034 28794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:04.101220 28794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:04.103489 28794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "87fa6165200f461ba893f3c5a18f4de2"
format_stamp: "Formatted at 2026-08-07 15:19:04 on dist-test-slave-gkw7"
server_key: "8815ec0bbc2e0ed09631bb91b20722fd"
server_key_iv: "9d6a78e9f0dfd22a7ba13ad05f117854"
server_key_version: "encryptionkey@0"
I20260807 15:19:04.104070 28794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "87fa6165200f461ba893f3c5a18f4de2"
format_stamp: "Formatted at 2026-08-07 15:19:04 on dist-test-slave-gkw7"
server_key: "8815ec0bbc2e0ed09631bb91b20722fd"
server_key_iv: "9d6a78e9f0dfd22a7ba13ad05f117854"
server_key_version: "encryptionkey@0"
I20260807 15:19:04.108143 28794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:19:04.110885 28816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:04.112398 28794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 15:19:04.112520 28794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "87fa6165200f461ba893f3c5a18f4de2"
format_stamp: "Formatted at 2026-08-07 15:19:04 on dist-test-slave-gkw7"
server_key: "8815ec0bbc2e0ed09631bb91b20722fd"
server_key_iv: "9d6a78e9f0dfd22a7ba13ad05f117854"
server_key_version: "encryptionkey@0"
I20260807 15:19:04.112613 28794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:04.136670 28794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:04.140097 28794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:04.140288 28794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:04.148523 28794 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34411
I20260807 15:19:04.148551 28890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34411 every 8 connection(s)
I20260807 15:19:04.149629 28794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:04.152702 28891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:04.153159 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 28794
I20260807 15:19:04.153329 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:04.153693 32667 external_mini_cluster.cc:1428] Setting key a23fc621960424fabc1b91bb982d08d7
I20260807 15:19:04.159631 28891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2: Bootstrap starting.
I20260807 15:19:04.162724 28891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2: Neither blocks nor log segments found. Creating new log.
Aug 07 15:19:04 dist-test-slave-gkw7 krb5kdc[28772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:19:04.163625 28891 log.cc:824] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:04.166354 28891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2: No bootstrap required, opened a new log
I20260807 15:19:04.169831 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:04.155278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43133 (local address 127.0.0.1:34411)
0807 15:19:04.155756 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:04.155772 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:04.155810 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:19:04.156524 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:04.156543 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:04.156917 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:04.157218 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:04.158323 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:04.159358 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:04.160522 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:04.161020 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:04.161147 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:04.164293 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:04.164298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:04.164312 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:04.164340 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:04.166340 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:04.166916 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:04.166923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:04.166929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:04.167078 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:04.167407 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:04.167411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:04.167413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:04.167941 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:04.168155 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:04.168418 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 15:19:04.168711 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":155,"threads_started":1}
I20260807 15:19:04.171244 28891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa6165200f461ba893f3c5a18f4de2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:19:04.171478 28891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:04.171538 28891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fa6165200f461ba893f3c5a18f4de2, State: Initialized, Role: FOLLOWER
I20260807 15:19:04.172049 28891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa6165200f461ba893f3c5a18f4de2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:19:04.172209 28891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:04.172251 28891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:04.172318 28891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:04.173094 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:04.173501 28891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa6165200f461ba893f3c5a18f4de2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:19:04.173905 28891 leader_election.cc:302] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87fa6165200f461ba893f3c5a18f4de2; no voters:
I20260807 15:19:04.174235 28891 leader_election.cc:288] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:04.174379 28899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:04.174638 28899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [term 1 LEADER]: Becoming Leader. State: Replica: 87fa6165200f461ba893f3c5a18f4de2, State: Running, Role: LEADER
I20260807 15:19:04.174963 28899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa6165200f461ba893f3c5a18f4de2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:19:04.175689 28891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:04.176251 28903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87fa6165200f461ba893f3c5a18f4de2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa6165200f461ba893f3c5a18f4de2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } } }
I20260807 15:19:04.176385 28903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:04.176824 28911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:04.177181 28906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87fa6165200f461ba893f3c5a18f4de2. Latest consensus state: current_term: 1 leader_uuid: "87fa6165200f461ba893f3c5a18f4de2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa6165200f461ba893f3c5a18f4de2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } } }
I20260807 15:19:04.177279 28906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:04.180447 28911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:04.187517 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:04.175470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38165 (local address 127.0.0.1:34411)
0807 15:19:04.175580 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:04.175585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:04.176110 (+ 525us) server_negotiation.cc:407] Connection header received
0807 15:19:04.177603 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:04.177607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:04.177658 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:04.177713 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:04.178743 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:04.179628 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:04.180817 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:04.181173 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:04.181258 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:04.183458 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:04.183462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:04.183465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:04.183502 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:04.185065 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:04.185625 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:04.185629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:04.185631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:04.185720 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:04.186091 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:04.186101 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:04.186102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:04.186291 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:04.186458 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:04.187178 (+ 720us) server_negotiation.cc:299] Negotiation successful
0807 15:19:04.187312 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:19:04.188320 28911 catalog_manager.cc:1347] Generated new cluster ID: 4a614c69412a4cd7b3cddb5492c88335
I20260807 15:19:04.188388 28911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:04.219072 28911 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:04.220157 28911 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:04.226421 28911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87fa6165200f461ba893f3c5a18f4de2: Generated new TSK 0
I20260807 15:19:04.227173 28911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:04.235296 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 28794
W20260807 15:19:04.247694 28770 connection.cc:535] client connection to 127.0.0.1:34411 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34411): Connection reset by peer (error 104)
2026-08-07T15:19:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:04 dist-test-slave-gkw7 krb5kdc[28935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:04 dist-test-slave-gkw7 krb5kdc[28935](info): set up 2 sockets
Aug 07 15:19:04 dist-test-slave-gkw7 krb5kdc[28935](info): commencing operation
krb5kdc: starting...
W20260807 15:19:05.473228 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:05 dist-test-slave-gkw7 krb5kdc[28935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:05.628688 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:05.746222 28953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:05.746518 28953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:05.746562 28953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:05.750618 28953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:05.750681 28953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:05.750701 28953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:05.750718 28953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:05.750733 28953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:05.756146 28953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44845
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:05.757210 28953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:05.758231 28953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:05.767454 28961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:05.767478 28960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:05.767462 28964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:05.767755 28953 server_base.cc:1034] running on GCE node
I20260807 15:19:05.768457 28953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:05.769582 28953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:05.770761 28953 hybrid_clock.cc:648] HybridClock initialized: now 1786115945770747 us; error 30 us; skew 500 ppm
Aug 07 15:19:05 dist-test-slave-gkw7 krb5kdc[28935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115945, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:05.773778 28953 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:05.774690 28953 webserver.cc:460] Webserver started at http://127.0.0.1:36863/ using document root <none> and password file <none>
I20260807 15:19:05.775297 28953 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:05.775419 28953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:05.775652 28953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:05.777848 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "d9819c512c5245ecbd093142d5ac9e81"
format_stamp: "Formatted at 2026-08-07 15:19:05 on dist-test-slave-gkw7"
server_key: "edc9c6c8c7088aa3c14134674bb9c7ce"
server_key_iv: "bb61ab7808753f76f3d24d58a0e77f52"
server_key_version: "encryptionkey@0"
I20260807 15:19:05.778429 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "d9819c512c5245ecbd093142d5ac9e81"
format_stamp: "Formatted at 2026-08-07 15:19:05 on dist-test-slave-gkw7"
server_key: "edc9c6c8c7088aa3c14134674bb9c7ce"
server_key_iv: "bb61ab7808753f76f3d24d58a0e77f52"
server_key_version: "encryptionkey@0"
I20260807 15:19:05.782395 28953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:19:05.784869 28973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:05.786262 28953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:19:05.786401 28953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "d9819c512c5245ecbd093142d5ac9e81"
format_stamp: "Formatted at 2026-08-07 15:19:05 on dist-test-slave-gkw7"
server_key: "edc9c6c8c7088aa3c14134674bb9c7ce"
server_key_iv: "bb61ab7808753f76f3d24d58a0e77f52"
server_key_version: "encryptionkey@0"
I20260807 15:19:05.786504 28953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:05.813275 28953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:05.817121 28953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:05.817337 28953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:05.824256 28953 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44845
I20260807 15:19:05.824338 29046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44845 every 8 connection(s)
I20260807 15:19:05.825241 28953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:05.826305 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 28953
I20260807 15:19:05.826442 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:05.826735 32667 external_mini_cluster.cc:1428] Setting key c7e3ece2ed22a089eb6b1e4d6193ede4
I20260807 15:19:05.829334 29048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:05.835759 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:05.828086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33735 (local address 127.0.0.1:44845)
0807 15:19:05.828482 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:05.828497 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:05.828531 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:19:05.829179 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:05.829195 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:05.829665 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:05.829961 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:05.830324 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:05.831324 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:05.832534 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:05.832994 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:05.833122 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:05.834088 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:05.834093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:05.834105 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:05.834474 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:05.834527 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:05.834549 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:05.834658 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:05.834864 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 15:19:05.835023 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
I20260807 15:19:05.836054 29048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81: Bootstrap starting.
I20260807 15:19:05.838665 29048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:05.839030 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:05.839542 29048 log.cc:824] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:05.841764 29048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81: No bootstrap required, opened a new log
I20260807 15:19:05.844890 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:05.841073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60911 (local address 127.0.0.1:44845)
0807 15:19:05.841198 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:05.841203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:05.841397 (+ 194us) server_negotiation.cc:407] Connection header received
0807 15:19:05.841467 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:05.841470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:05.841515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:05.841632 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:05.841902 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:05.842403 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:05.843234 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:05.843458 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:05.843568 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:05.844422 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:05.844425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:05.844427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:05.844438 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:05.844461 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:05.844471 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:05.844568 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:05.844720 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 15:19:05.844759 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:19:05.845661 29048 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9819c512c5245ecbd093142d5ac9e81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44845 } }
I20260807 15:19:05.845974 29048 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:05.846051 29048 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9819c512c5245ecbd093142d5ac9e81, State: Initialized, Role: FOLLOWER
I20260807 15:19:05.846459 29048 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9819c512c5245ecbd093142d5ac9e81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44845 } }
I20260807 15:19:05.846612 29048 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:05.846683 29048 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:05.846789 29048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:05.847697 29048 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9819c512c5245ecbd093142d5ac9e81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44845 } }
I20260807 15:19:05.848059 29048 leader_election.cc:302] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9819c512c5245ecbd093142d5ac9e81; no voters:
I20260807 15:19:05.848389 29048 leader_election.cc:288] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:05.848533 29059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:05.848832 29059 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [term 1 LEADER]: Becoming Leader. State: Replica: d9819c512c5245ecbd093142d5ac9e81, State: Running, Role: LEADER
I20260807 15:19:05.849206 29059 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9819c512c5245ecbd093142d5ac9e81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44845 } }
I20260807 15:19:05.849771 29048 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:05.851023 29062 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9819c512c5245ecbd093142d5ac9e81. Latest consensus state: current_term: 1 leader_uuid: "d9819c512c5245ecbd093142d5ac9e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9819c512c5245ecbd093142d5ac9e81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44845 } } }
I20260807 15:19:05.851164 29062 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:05.851568 29060 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9819c512c5245ecbd093142d5ac9e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9819c512c5245ecbd093142d5ac9e81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44845 } } }
I20260807 15:19:05.851693 29060 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:05.851626 29072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:05.854736 29072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:05.860078 29072 catalog_manager.cc:1347] Generated new cluster ID: 5bac1f4b433744878232896cb6913e45
I20260807 15:19:05.860150 29072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:05.871968 29072 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:05.873199 29072 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:05.884456 29072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9819c512c5245ecbd093142d5ac9e81: Generated new TSK 0
I20260807 15:19:05.885358 29072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:05.889742 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 28953
W20260807 15:19:05.907809 28931 connection.cc:535] client connection to 127.0.0.1:44845 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44845): Connection reset by peer (error 104)
2026-08-07T15:19:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1660 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:05 dist-test-slave-gkw7 krb5kdc[29089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:05 dist-test-slave-gkw7 krb5kdc[29089](info): set up 2 sockets
Aug 07 15:19:05 dist-test-slave-gkw7 krb5kdc[29089](info): commencing operation
krb5kdc: starting...
W20260807 15:19:07.190439 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:07 dist-test-slave-gkw7 krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:07.330575 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:07.448660 29116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:07.448952 29116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:07.448995 29116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:07.453135 29116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:07.453202 29116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:07.453222 29116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:07.453238 29116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:07.453253 29116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:07.458755 29116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:07.459841 29116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:07.460897 29116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:19:07.469770 29116 server_base.cc:1034] running on GCE node
W20260807 15:19:07.469741 29128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:07.469718 29124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:07.469707 29122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:07.470721 29116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:07.471930 29116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:07.473145 29116 hybrid_clock.cc:648] HybridClock initialized: now 1786115947473113 us; error 43 us; skew 500 ppm
Aug 07 15:19:07 dist-test-slave-gkw7 krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115947, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:07.482170 29116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:07.483140 29116 webserver.cc:460] Webserver started at http://127.0.0.1:37115/ using document root <none> and password file <none>
I20260807 15:19:07.483865 29116 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:07.483965 29116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:07.484203 29116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:07.486439 29116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "9a0a468af09340d9b644cafaf7db64ff"
format_stamp: "Formatted at 2026-08-07 15:19:07 on dist-test-slave-gkw7"
server_key: "e6eec87eb6bb58e61705b0f7a28cac8e"
server_key_iv: "e39e3b8a8e6789b0abe3461f54b00a98"
server_key_version: "encryptionkey@0"
I20260807 15:19:07.487030 29116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "9a0a468af09340d9b644cafaf7db64ff"
format_stamp: "Formatted at 2026-08-07 15:19:07 on dist-test-slave-gkw7"
server_key: "e6eec87eb6bb58e61705b0f7a28cac8e"
server_key_iv: "e39e3b8a8e6789b0abe3461f54b00a98"
server_key_version: "encryptionkey@0"
I20260807 15:19:07.491052 29116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:19:07.493834 29136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:07.495116 29116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:19:07.495276 29116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "9a0a468af09340d9b644cafaf7db64ff"
format_stamp: "Formatted at 2026-08-07 15:19:07 on dist-test-slave-gkw7"
server_key: "e6eec87eb6bb58e61705b0f7a28cac8e"
server_key_iv: "e39e3b8a8e6789b0abe3461f54b00a98"
server_key_version: "encryptionkey@0"
I20260807 15:19:07.495399 29116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:07.500588 29116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:07.503680 29116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:07.503885 29116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:07.511842 29116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44977
I20260807 15:19:07.511861 29219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44977 every 8 connection(s)
I20260807 15:19:07.512817 29116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:07.515970 29220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:07.518168 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 29116
I20260807 15:19:07.518338 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:07.518693 32667 external_mini_cluster.cc:1428] Setting key ccc4e2549c9172cc3d2f9add88a686a4
I20260807 15:19:07.522230 29220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff: Bootstrap starting.
I20260807 15:19:07.524756 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:07.520260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56849 (local address 127.0.0.1:44977)
0807 15:19:07.520774 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:07.520792 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:07.520832 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:19:07.521626 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:07.521649 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:07.522057 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:07.522441 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:07.523399 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:07.523404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:07.523416 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:07.523429 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:07.523492 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:07.523524 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:07.523706 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:07.523791 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 15:19:07.523974 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":159,"threads_started":1}
I20260807 15:19:07.524765 29220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:07.525870 29220 log.cc:824] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:07.528280 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:07.528477 29220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff: No bootstrap required, opened a new log
I20260807 15:19:07.532454 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:07.530541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47629 (local address 127.0.0.1:44977)
0807 15:19:07.530664 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:07.530669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:07.530857 (+ 188us) server_negotiation.cc:407] Connection header received
0807 15:19:07.531021 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:07.531030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:07.531062 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:07.531151 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:07.532017 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:07.532021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:07.532024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:07.532035 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:07.532069 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:07.532082 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:07.532178 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:07.532281 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 15:19:07.532317 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:19:07.533362 29220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0a468af09340d9b644cafaf7db64ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260807 15:19:07.533742 29220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:07.533838 29220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0a468af09340d9b644cafaf7db64ff, State: Initialized, Role: FOLLOWER
I20260807 15:19:07.534335 29220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0a468af09340d9b644cafaf7db64ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260807 15:19:07.534480 29220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:07.534559 29220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:07.534703 29220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:07.535836 29220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0a468af09340d9b644cafaf7db64ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260807 15:19:07.536244 29220 leader_election.cc:302] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a0a468af09340d9b644cafaf7db64ff; no voters:
I20260807 15:19:07.536610 29220 leader_election.cc:288] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:07.536784 29234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:07.537071 29234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [term 1 LEADER]: Becoming Leader. State: Replica: 9a0a468af09340d9b644cafaf7db64ff, State: Running, Role: LEADER
I20260807 15:19:07.537428 29234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0a468af09340d9b644cafaf7db64ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260807 15:19:07.538251 29220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:07.539479 29236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a0a468af09340d9b644cafaf7db64ff. Latest consensus state: current_term: 1 leader_uuid: "9a0a468af09340d9b644cafaf7db64ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0a468af09340d9b644cafaf7db64ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } } }
I20260807 15:19:07.539601 29236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:07.540469 29244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:07.540517 29235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a0a468af09340d9b644cafaf7db64ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0a468af09340d9b644cafaf7db64ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } } }
I20260807 15:19:07.540633 29235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:07.544059 29244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:07.550050 29244 catalog_manager.cc:1347] Generated new cluster ID: 9bf77845320748979ad63c3e8a23f96f
I20260807 15:19:07.550146 29244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:07.589437 29244 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:07.589881 29244 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:07.609577 29244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a0a468af09340d9b644cafaf7db64ff: Generated new TSK 0
I20260807 15:19:07.610386 29244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:07.626834 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 29116
2026-08-07T15:19:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1737 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:07 dist-test-slave-gkw7 krb5kdc[29259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:07 dist-test-slave-gkw7 krb5kdc[29259](info): set up 2 sockets
Aug 07 15:19:07 dist-test-slave-gkw7 krb5kdc[29259](info): commencing operation
krb5kdc: starting...
W20260807 15:19:08.857625 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:08 dist-test-slave-gkw7 krb5kdc[29259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:08.997728 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:09.112661 29275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:09.112967 29275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:09.113058 29275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:09.117103 29275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:09.117216 29275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:09.117252 29275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:09.117303 29275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:09.117360 29275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:09.122988 29275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:09.124202 29275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:09.125329 29275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:09.134298 29285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:09.134380 29291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:09.134315 29287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:09.134572 29275 server_base.cc:1034] running on GCE node
I20260807 15:19:09.135167 29275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:09.136286 29275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:09.137476 29275 hybrid_clock.cc:648] HybridClock initialized: now 1786115949137453 us; error 39 us; skew 500 ppm
Aug 07 15:19:09 dist-test-slave-gkw7 krb5kdc[29259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:09.140244 29275 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:09.141131 29275 webserver.cc:460] Webserver started at http://127.0.0.1:38251/ using document root <none> and password file <none>
I20260807 15:19:09.141709 29275 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:09.141788 29275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:09.141959 29275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:09.143956 29275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "09ee8e0ed780438591217fbbec43acfd"
format_stamp: "Formatted at 2026-08-07 15:19:09 on dist-test-slave-gkw7"
server_key: "8262381b54acaeed0a9d75570ca5b425"
server_key_iv: "0e90e4c2e169bda5542b705c87bf69be"
server_key_version: "encryptionkey@0"
I20260807 15:19:09.144542 29275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "09ee8e0ed780438591217fbbec43acfd"
format_stamp: "Formatted at 2026-08-07 15:19:09 on dist-test-slave-gkw7"
server_key: "8262381b54acaeed0a9d75570ca5b425"
server_key_iv: "0e90e4c2e169bda5542b705c87bf69be"
server_key_version: "encryptionkey@0"
I20260807 15:19:09.148412 29275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:19:09.151127 29299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:09.152412 29275 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:09.152580 29275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "09ee8e0ed780438591217fbbec43acfd"
format_stamp: "Formatted at 2026-08-07 15:19:09 on dist-test-slave-gkw7"
server_key: "8262381b54acaeed0a9d75570ca5b425"
server_key_iv: "0e90e4c2e169bda5542b705c87bf69be"
server_key_version: "encryptionkey@0"
I20260807 15:19:09.152704 29275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:09.186498 29275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:09.189880 29275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:09.190102 29275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:09.197496 29275 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35137
I20260807 15:19:09.197551 29378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35137 every 8 connection(s)
I20260807 15:19:09.198468 29275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:09.201153 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:09.205473 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 29275
I20260807 15:19:09.205689 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:09.205992 32667 external_mini_cluster.cc:1428] Setting key a84812317e8684c720b75f7d268f9e0f
I20260807 15:19:09.207103 29379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd: Bootstrap starting.
I20260807 15:19:09.209820 29379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:09.210686 29379 log.cc:824] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:09.212715 29379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd: No bootstrap required, opened a new log
Aug 07 15:19:09 dist-test-slave-gkw7 krb5kdc[29259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:19:09.216552 29379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ee8e0ed780438591217fbbec43acfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35137 } }
I20260807 15:19:09.216835 29379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:09.216892 29379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09ee8e0ed780438591217fbbec43acfd, State: Initialized, Role: FOLLOWER
I20260807 15:19:09.217407 29379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ee8e0ed780438591217fbbec43acfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35137 } }
I20260807 15:19:09.217599 29379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:09.217664 29379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:09.217772 29379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:09.218911 29379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ee8e0ed780438591217fbbec43acfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35137 } }
I20260807 15:19:09.219293 29379 leader_election.cc:302] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09ee8e0ed780438591217fbbec43acfd; no voters:
I20260807 15:19:09.219676 29379 leader_election.cc:288] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:09.219820 29387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:09.221051 29379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:09.221920 29387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [term 1 LEADER]: Becoming Leader. State: Replica: 09ee8e0ed780438591217fbbec43acfd, State: Running, Role: LEADER
I20260807 15:19:09.222295 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:09.207507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57545 (local address 127.0.0.1:35137)
0807 15:19:09.207947 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:09.207966 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:09.208009 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:19:09.208884 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:09.208908 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:09.209332 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:09.209729 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:09.210515 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:09.211897 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:09.212809 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:09.213472 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:09.213679 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:09.216159 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:09.216165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:09.216182 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:09.216212 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:09.218366 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:09.218786 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:09.218795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:09.218803 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:09.218958 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:09.219273 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:09.219276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:09.219278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:09.219847 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:09.220768 (+ 921us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:09.220916 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:19:09.221220 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":124,"threads_started":1}
I20260807 15:19:09.222297 29387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ee8e0ed780438591217fbbec43acfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35137 } }
I20260807 15:19:09.223636 29390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09ee8e0ed780438591217fbbec43acfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ee8e0ed780438591217fbbec43acfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35137 } } }
I20260807 15:19:09.223817 29390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:09.223908 29389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09ee8e0ed780438591217fbbec43acfd. Latest consensus state: current_term: 1 leader_uuid: "09ee8e0ed780438591217fbbec43acfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ee8e0ed780438591217fbbec43acfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35137 } } }
I20260807 15:19:09.225538 29389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:09.226572 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 15:19:09.227193 29412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:09.227283 29412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:09.227417 29413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:09.228287 29413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:09.234421 29413 catalog_manager.cc:1347] Generated new cluster ID: 1f0d8c6f0cb145bd911313b2ff1e6c18
I20260807 15:19:09.234493 29413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:09.242316 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:09.228869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36439 (local address 127.0.0.1:35137)
0807 15:19:09.228979 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:09.228983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:09.229892 (+ 909us) server_negotiation.cc:407] Connection header received
0807 15:19:09.230014 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:09.230018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:09.230068 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:09.230162 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:09.231007 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:09.232290 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:09.233419 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:09.233754 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:09.233872 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:09.238690 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:09.238694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:09.238697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:09.238726 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:09.240497 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:09.240977 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:09.240981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:09.240983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:09.241075 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:09.241352 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:09.241356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:09.241357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:09.241621 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:09.241752 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:09.242013 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 15:19:09.242146 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:19:09.256621 29413 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:09.257990 29413 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:09.269954 29413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09ee8e0ed780438591217fbbec43acfd: Generated new TSK 0
I20260807 15:19:09.270821 29413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:09.279898 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 29275
2026-08-07T15:19:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1645 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:09 dist-test-slave-gkw7 krb5kdc[29428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:09 dist-test-slave-gkw7 krb5kdc[29428](info): set up 2 sockets
Aug 07 15:19:09 dist-test-slave-gkw7 krb5kdc[29428](info): commencing operation
krb5kdc: starting...
W20260807 15:19:10.508953 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:10 dist-test-slave-gkw7 krb5kdc[29428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:10.646771 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:10.764587 29445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:10.764860 29445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:10.764901 29445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:10.768816 29445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:10.768877 29445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:10.768896 29445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:10.768913 29445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:10.768927 29445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:10.774155 29445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:10.775153 29445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:10.776155 29445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:10.784698 29457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:10.784693 29454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:10.784761 29453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:10.784919 29445 server_base.cc:1034] running on GCE node
I20260807 15:19:10.785784 29445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:10.786911 29445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:10.788136 29445 hybrid_clock.cc:648] HybridClock initialized: now 1786115950788107 us; error 49 us; skew 500 ppm
Aug 07 15:19:10 dist-test-slave-gkw7 krb5kdc[29428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:10.791225 29445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:10.792263 29445 webserver.cc:460] Webserver started at http://127.0.0.1:35667/ using document root <none> and password file <none>
I20260807 15:19:10.792907 29445 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:10.793005 29445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:10.793239 29445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:10.795416 29445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "c1b9b301ab564638a4ce43d537a6c255"
format_stamp: "Formatted at 2026-08-07 15:19:10 on dist-test-slave-gkw7"
server_key: "5e7558e16aaef68adfddd823c0cb15ed"
server_key_iv: "9fc5ed1c8e52f761d9c9d1c3733c9172"
server_key_version: "encryptionkey@0"
I20260807 15:19:10.796010 29445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "c1b9b301ab564638a4ce43d537a6c255"
format_stamp: "Formatted at 2026-08-07 15:19:10 on dist-test-slave-gkw7"
server_key: "5e7558e16aaef68adfddd823c0cb15ed"
server_key_iv: "9fc5ed1c8e52f761d9c9d1c3733c9172"
server_key_version: "encryptionkey@0"
I20260807 15:19:10.799952 29445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:19:10.802573 29465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:10.803815 29445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:19:10.803962 29445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "c1b9b301ab564638a4ce43d537a6c255"
format_stamp: "Formatted at 2026-08-07 15:19:10 on dist-test-slave-gkw7"
server_key: "5e7558e16aaef68adfddd823c0cb15ed"
server_key_iv: "9fc5ed1c8e52f761d9c9d1c3733c9172"
server_key_version: "encryptionkey@0"
I20260807 15:19:10.804077 29445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:10.817396 29445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:10.820802 29445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:10.821015 29445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:10.828502 29445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33041
I20260807 15:19:10.828518 29543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33041 every 8 connection(s)
I20260807 15:19:10.829478 29445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:10.832278 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:10.834741 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 29445
I20260807 15:19:10.834890 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:10.835237 32667 external_mini_cluster.cc:1428] Setting key 745f72cb4084dca0f5f7f209eae13fc7
I20260807 15:19:10.839572 29546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255: Bootstrap starting.
I20260807 15:19:10.842258 29546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:10.843086 29546 log.cc:824] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:10.844417 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:10.836682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41957 (local address 127.0.0.1:33041)
0807 15:19:10.837081 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:10.837100 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:10.837151 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:19:10.837872 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:10.837891 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:10.838228 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:10.838501 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:10.838870 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:10.839889 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:10.841073 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:10.841605 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:10.841738 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:10.842659 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:10.842665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:10.842678 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:10.843057 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:10.843111 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:10.843144 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:10.843269 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:10.843405 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 15:19:10.843631 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":122,"threads_started":1}
I20260807 15:19:10.845108 29546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255: No bootstrap required, opened a new log
I20260807 15:19:10.847350 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:10.849157 29546 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b9b301ab564638a4ce43d537a6c255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260807 15:19:10.849519 29546 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:10.849607 29546 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b9b301ab564638a4ce43d537a6c255, State: Initialized, Role: FOLLOWER
I20260807 15:19:10.850051 29546 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b9b301ab564638a4ce43d537a6c255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260807 15:19:10.850211 29546 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:10.850282 29546 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:10.850379 29546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:10.851298 29546 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b9b301ab564638a4ce43d537a6c255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260807 15:19:10.851630 29546 leader_election.cc:302] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1b9b301ab564638a4ce43d537a6c255; no voters:
I20260807 15:19:10.851955 29546 leader_election.cc:288] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:10.852025 29560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:10.852236 29560 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [term 1 LEADER]: Becoming Leader. State: Replica: c1b9b301ab564638a4ce43d537a6c255, State: Running, Role: LEADER
I20260807 15:19:10.852582 29560 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b9b301ab564638a4ce43d537a6c255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260807 15:19:10.853260 29546 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:10.853554 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:10.849360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38537 (local address 127.0.0.1:33041)
0807 15:19:10.849489 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:10.849494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:10.849781 (+ 287us) server_negotiation.cc:407] Connection header received
0807 15:19:10.849815 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:10.849817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:10.849851 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:10.849938 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:10.850199 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:10.850804 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:10.851667 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:10.851981 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:10.852094 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:10.853091 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:10.853094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:10.853097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:10.853109 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:10.853137 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:10.853153 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:10.853234 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:10.853326 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 15:19:10.853374 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:19:10.853876 29560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1b9b301ab564638a4ce43d537a6c255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b9b301ab564638a4ce43d537a6c255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260807 15:19:10.854343 29561 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1b9b301ab564638a4ce43d537a6c255. Latest consensus state: current_term: 1 leader_uuid: "c1b9b301ab564638a4ce43d537a6c255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b9b301ab564638a4ce43d537a6c255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260807 15:19:10.854666 29561 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:10.854914 29560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:10.855105 29573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:10.858062 29573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:10.864236 29573 catalog_manager.cc:1347] Generated new cluster ID: 28bc7bb3f88741c5b254954b2bd366fd
I20260807 15:19:10.864313 29573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:10.876087 29573 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:10.877104 29573 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:10.881692 29573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1b9b301ab564638a4ce43d537a6c255: Generated new TSK 0
I20260807 15:19:10.882369 29573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:10.889297 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 29445
2026-08-07T15:19:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:10 dist-test-slave-gkw7 krb5kdc[29592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:10 dist-test-slave-gkw7 krb5kdc[29592](info): set up 2 sockets
Aug 07 15:19:10 dist-test-slave-gkw7 krb5kdc[29592](info): commencing operation
krb5kdc: starting...
W20260807 15:19:12.155083 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:12 dist-test-slave-gkw7 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:12.288452 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:12.404268 29615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:12.404587 29615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:12.404702 29615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:12.409149 29615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:12.409267 29615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:12.409308 29615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:12.409366 29615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:12.409423 29615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:12.415201 29615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:12.416329 29615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:12.417376 29615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:12.426306 29625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:12.426317 29626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:12.426306 29629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:12.426554 29615 server_base.cc:1034] running on GCE node
I20260807 15:19:12.427211 29615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:12.428337 29615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:12.429515 29615 hybrid_clock.cc:648] HybridClock initialized: now 1786115952429508 us; error 33 us; skew 500 ppm
Aug 07 15:19:12 dist-test-slave-gkw7 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115952, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:12.432669 29615 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:12.433768 29615 webserver.cc:460] Webserver started at http://127.0.0.1:36761/ using document root <none> and password file <none>
I20260807 15:19:12.434342 29615 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:12.434408 29615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:12.434607 29615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:12.436798 29615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "35bf6567224e4181bd09ac38f486a80a"
format_stamp: "Formatted at 2026-08-07 15:19:12 on dist-test-slave-gkw7"
server_key: "e075b31b9af3094b2075f10e636b4e06"
server_key_iv: "158d6748953c3be3ee4e92e885aaccc2"
server_key_version: "encryptionkey@0"
I20260807 15:19:12.437390 29615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "35bf6567224e4181bd09ac38f486a80a"
format_stamp: "Formatted at 2026-08-07 15:19:12 on dist-test-slave-gkw7"
server_key: "e075b31b9af3094b2075f10e636b4e06"
server_key_iv: "158d6748953c3be3ee4e92e885aaccc2"
server_key_version: "encryptionkey@0"
I20260807 15:19:12.441465 29615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260807 15:19:12.444279 29640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:12.445719 29615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:19:12.445878 29615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "35bf6567224e4181bd09ac38f486a80a"
format_stamp: "Formatted at 2026-08-07 15:19:12 on dist-test-slave-gkw7"
server_key: "e075b31b9af3094b2075f10e636b4e06"
server_key_iv: "158d6748953c3be3ee4e92e885aaccc2"
server_key_version: "encryptionkey@0"
I20260807 15:19:12.446007 29615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:12.451208 29615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:12.454528 29615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:12.454754 29615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:12.463088 29615 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36055
I20260807 15:19:12.463135 29712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36055 every 8 connection(s)
I20260807 15:19:12.464121 29615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:12.465945 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 29615
I20260807 15:19:12.466130 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:12.466455 32667 external_mini_cluster.cc:1428] Setting key ca5f9931b0d923610a5fdb244941642c
I20260807 15:19:12.467185 29713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:12.472555 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:12.467981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37977 (local address 127.0.0.1:36055)
0807 15:19:12.468444 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:12.468460 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:12.468708 (+ 248us) server_negotiation.cc:407] Connection header received
0807 15:19:12.469517 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:12.469584 (+ 67us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:12.469915 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:12.470214 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:12.471131 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:12.471136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:12.471148 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:12.471160 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:12.471217 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:12.471238 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:12.471359 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:12.471615 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 15:19:12.471816 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":148,"threads_started":1}
I20260807 15:19:12.474754 29713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a: Bootstrap starting.
I20260807 15:19:12.475657 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:12.477191 29713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:12.478092 29713 log.cc:824] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:12.479586 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:12.477866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47153 (local address 127.0.0.1:36055)
0807 15:19:12.477986 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:12.477992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:12.478231 (+ 239us) server_negotiation.cc:407] Connection header received
0807 15:19:12.478318 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:12.478321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:12.478365 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:12.478440 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:12.479230 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:12.479234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:12.479237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:12.479249 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:12.479276 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:12.479289 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:12.479379 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:12.479441 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:19:12.479477 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:19:12.480342 29713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a: No bootstrap required, opened a new log
I20260807 15:19:12.483906 29713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf6567224e4181bd09ac38f486a80a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36055 } }
I20260807 15:19:12.484125 29713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:12.484200 29713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bf6567224e4181bd09ac38f486a80a, State: Initialized, Role: FOLLOWER
I20260807 15:19:12.484652 29713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf6567224e4181bd09ac38f486a80a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36055 } }
I20260807 15:19:12.484773 29713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:12.484814 29713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:12.484920 29713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:12.486024 29713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf6567224e4181bd09ac38f486a80a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36055 } }
I20260807 15:19:12.486371 29713 leader_election.cc:302] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35bf6567224e4181bd09ac38f486a80a; no voters:
I20260807 15:19:12.486685 29713 leader_election.cc:288] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:12.486855 29729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:12.487139 29729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [term 1 LEADER]: Becoming Leader. State: Replica: 35bf6567224e4181bd09ac38f486a80a, State: Running, Role: LEADER
I20260807 15:19:12.487471 29729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf6567224e4181bd09ac38f486a80a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36055 } }
I20260807 15:19:12.487965 29713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:12.489239 29731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35bf6567224e4181bd09ac38f486a80a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf6567224e4181bd09ac38f486a80a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36055 } } }
I20260807 15:19:12.489369 29731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:12.489722 29734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35bf6567224e4181bd09ac38f486a80a. Latest consensus state: current_term: 1 leader_uuid: "35bf6567224e4181bd09ac38f486a80a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf6567224e4181bd09ac38f486a80a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36055 } } }
I20260807 15:19:12.489895 29734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a [sys.catalog]: This master's current role is: LEADER
W20260807 15:19:12.493043 29758 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:12.493139 29758 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:12.493214 29746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:12.494004 29746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:12.499871 29746 catalog_manager.cc:1347] Generated new cluster ID: ebea6e58d2614d148cd86e5eea513450
I20260807 15:19:12.499939 29746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:12.529771 29746 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:12.530166 29746 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:12.539547 29746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35bf6567224e4181bd09ac38f486a80a: Generated new TSK 0
I20260807 15:19:12.540457 29746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:12.549587 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 29615
2026-08-07T15:19:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1663 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:12 dist-test-slave-gkw7 krb5kdc[29773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:12 dist-test-slave-gkw7 krb5kdc[29773](info): set up 2 sockets
Aug 07 15:19:12 dist-test-slave-gkw7 krb5kdc[29773](info): commencing operation
krb5kdc: starting...
W20260807 15:19:13.771840 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:13 dist-test-slave-gkw7 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:13.904832 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:14.022009 29800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:14.022297 29800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:14.022372 29800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:14.026413 29800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:14.026474 29800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:14.026494 29800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:14.026537 29800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:14.026567 29800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:14.031898 29800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:14.033016 29800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:14.034148 29800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:14.042577 29815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:14.042704 29809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:14.042773 29811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:14.043004 29800 server_base.cc:1034] running on GCE node
I20260807 15:19:14.043503 29800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:14.044590 29800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:14.045753 29800 hybrid_clock.cc:648] HybridClock initialized: now 1786115954045739 us; error 26 us; skew 500 ppm
Aug 07 15:19:14 dist-test-slave-gkw7 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115954, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:14.048689 29800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:14.049633 29800 webserver.cc:460] Webserver started at http://127.0.0.1:46693/ using document root <none> and password file <none>
I20260807 15:19:14.050217 29800 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:14.050285 29800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:14.050501 29800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:14.052641 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "b2457277126f4bfbba19cd72301c7fb2"
format_stamp: "Formatted at 2026-08-07 15:19:14 on dist-test-slave-gkw7"
server_key: "928e9dcab369990069b78d80574a06a1"
server_key_iv: "5dda78c566756bf050cb308ae38eaf54"
server_key_version: "encryptionkey@0"
I20260807 15:19:14.053234 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "b2457277126f4bfbba19cd72301c7fb2"
format_stamp: "Formatted at 2026-08-07 15:19:14 on dist-test-slave-gkw7"
server_key: "928e9dcab369990069b78d80574a06a1"
server_key_iv: "5dda78c566756bf050cb308ae38eaf54"
server_key_version: "encryptionkey@0"
I20260807 15:19:14.057015 29800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:19:14.059593 29828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:14.060858 29800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:19:14.060998 29800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "b2457277126f4bfbba19cd72301c7fb2"
format_stamp: "Formatted at 2026-08-07 15:19:14 on dist-test-slave-gkw7"
server_key: "928e9dcab369990069b78d80574a06a1"
server_key_iv: "5dda78c566756bf050cb308ae38eaf54"
server_key_version: "encryptionkey@0"
I20260807 15:19:14.061112 29800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:14.081365 29800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:14.084599 29800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:14.084761 29800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:14.092175 29800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42409
I20260807 15:19:14.092195 29908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42409 every 8 connection(s)
I20260807 15:19:14.093145 29800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:14.096144 29909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:14.101737 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2: Bootstrap starting.
I20260807 15:19:14.102321 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 29800
I20260807 15:19:14.102471 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:14.102777 32667 external_mini_cluster.cc:1428] Setting key b8a4b7e09943b32a439da7aa7d602c8b
I20260807 15:19:14.104599 29909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:14.105353 29909 log.cc:824] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:14.107307 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2: No bootstrap required, opened a new log
I20260807 15:19:14.110908 29909 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2457277126f4bfbba19cd72301c7fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42409 } }
I20260807 15:19:14.111159 29909 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:14.111245 29909 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2457277126f4bfbba19cd72301c7fb2, State: Initialized, Role: FOLLOWER
Aug 07 15:19:14 dist-test-slave-gkw7 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:19:14.111773 29909 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2457277126f4bfbba19cd72301c7fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42409 } }
I20260807 15:19:14.111954 29909 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:14.112026 29909 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:14.112138 29909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:14.113279 29909 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2457277126f4bfbba19cd72301c7fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42409 } }
I20260807 15:19:14.113705 29909 leader_election.cc:302] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2457277126f4bfbba19cd72301c7fb2; no voters:
I20260807 15:19:14.114041 29909 leader_election.cc:288] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:14.114151 29915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:14.114434 29915 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [term 1 LEADER]: Becoming Leader. State: Replica: b2457277126f4bfbba19cd72301c7fb2, State: Running, Role: LEADER
I20260807 15:19:14.114848 29915 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2457277126f4bfbba19cd72301c7fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42409 } }
I20260807 15:19:14.115392 29909 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:14.116204 29915 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2457277126f4bfbba19cd72301c7fb2. Latest consensus state: current_term: 1 leader_uuid: "b2457277126f4bfbba19cd72301c7fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2457277126f4bfbba19cd72301c7fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42409 } } }
I20260807 15:19:14.116400 29915 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:14.116684 29918 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2457277126f4bfbba19cd72301c7fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2457277126f4bfbba19cd72301c7fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42409 } } }
I20260807 15:19:14.116780 29918 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:14.116901 29930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:14.119143 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:14.104055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:33245 (local address 127.0.0.1:42409)
0807 15:19:14.104515 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:14.104531 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:14.104567 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:19:14.105255 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:14.105274 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:14.105706 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:14.106048 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:14.106902 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:14.107939 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:14.108837 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:14.109328 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:14.109506 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:14.112380 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:14.112386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:14.112399 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:14.112425 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:14.114526 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:14.115254 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:14.115260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:14.115266 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:14.115412 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:14.115926 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:14.115930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:14.115931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:14.116531 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:14.117475 (+ 944us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:14.117744 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 15:19:14.118023 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":140,"threads_started":1}
I20260807 15:19:14.120520 29930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:14.122911 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:14.127245 29930 catalog_manager.cc:1347] Generated new cluster ID: a49fbf976c524ce7962270e51ec148c0
I20260807 15:19:14.127310 29930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:14.134796 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:14.125305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:46627 (local address 127.0.0.1:42409)
0807 15:19:14.125413 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:14.125418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:14.125764 (+ 346us) server_negotiation.cc:407] Connection header received
0807 15:19:14.125907 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:14.125910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:14.125957 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:14.126051 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:14.126746 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:14.127536 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:14.128732 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:14.129088 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:14.129192 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:14.130990 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:14.130993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:14.130996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:14.131022 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:14.132693 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:14.133258 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:14.133261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:14.133263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:14.133348 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:14.133733 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:14.133736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:14.133738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:14.133940 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:14.134082 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:14.134395 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 15:19:14.134647 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:19:14.136439 29930 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:14.137326 29930 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:14.142349 29930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2457277126f4bfbba19cd72301c7fb2: Generated new TSK 0
I20260807 15:19:14.143235 29930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:14.153810 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 29800
W20260807 15:19:14.168174 29765 connection.cc:535] client connection to 127.0.0.1:42409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42409): Connection reset by peer (error 104)
2026-08-07T15:19:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:14 dist-test-slave-gkw7 krb5kdc[29957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:14 dist-test-slave-gkw7 krb5kdc[29957](info): set up 2 sockets
Aug 07 15:19:14 dist-test-slave-gkw7 krb5kdc[29957](info): commencing operation
krb5kdc: starting...
W20260807 15:19:15.376178 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:15 dist-test-slave-gkw7 krb5kdc[29957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:15.506462 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:15.640324 29977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:15.640614 29977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:15.640656 29977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:15.644769 29977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:15.644841 29977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:15.644861 29977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:15.644877 29977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:15.644923 29977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:15.650282 29977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:15.651429 29977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:15.652521 29977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:15.661505 29983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:15.661468 29986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:15.661486 29989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:15.661737 29977 server_base.cc:1034] running on GCE node
I20260807 15:19:15.662400 29977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:15.663537 29977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:15.664750 29977 hybrid_clock.cc:648] HybridClock initialized: now 1786115955664735 us; error 27 us; skew 500 ppm
Aug 07 15:19:15 dist-test-slave-gkw7 krb5kdc[29957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:15.667672 29977 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:15.668592 29977 webserver.cc:460] Webserver started at http://127.0.0.1:41165/ using document root <none> and password file <none>
I20260807 15:19:15.669127 29977 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:15.669186 29977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:15.669359 29977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:15.671458 29977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "6d209a11df3c4e36b5b6b89e434e3ad3"
format_stamp: "Formatted at 2026-08-07 15:19:15 on dist-test-slave-gkw7"
server_key: "860232dfdb365b28c0bc5c0db3b9e9f5"
server_key_iv: "7917feba7f485fc7935481b45d3075e6"
server_key_version: "encryptionkey@0"
I20260807 15:19:15.671983 29977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "6d209a11df3c4e36b5b6b89e434e3ad3"
format_stamp: "Formatted at 2026-08-07 15:19:15 on dist-test-slave-gkw7"
server_key: "860232dfdb365b28c0bc5c0db3b9e9f5"
server_key_iv: "7917feba7f485fc7935481b45d3075e6"
server_key_version: "encryptionkey@0"
I20260807 15:19:15.675748 29977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:19:15.678308 29997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:15.679605 29977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:19:15.679755 29977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "6d209a11df3c4e36b5b6b89e434e3ad3"
format_stamp: "Formatted at 2026-08-07 15:19:15 on dist-test-slave-gkw7"
server_key: "860232dfdb365b28c0bc5c0db3b9e9f5"
server_key_iv: "7917feba7f485fc7935481b45d3075e6"
server_key_version: "encryptionkey@0"
I20260807 15:19:15.679865 29977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:15.708127 29977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:15.711566 29977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:15.711797 29977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:15.719174 29977 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43305
I20260807 15:19:15.719271 30079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43305 every 8 connection(s)
I20260807 15:19:15.720223 29977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:15.723223 30080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:15.724447 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 29977
I20260807 15:19:15.724591 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:15.724901 32667 external_mini_cluster.cc:1428] Setting key ac2818f5f11c7102ea9676279993c3df
I20260807 15:19:15.730126 30080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3: Bootstrap starting.
I20260807 15:19:15.732728 30080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:15.733667 30080 log.cc:824] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:15.735042 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:15.726450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:47013 (local address 127.0.0.1:43305)
0807 15:19:15.726886 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:15.726911 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:15.726955 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:19:15.727784 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:15.727810 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:15.728263 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:15.728626 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:15.729057 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:15.730533 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:15.731405 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:15.732067 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:15.732237 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:15.733121 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:15.733127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:15.733143 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:15.733665 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:15.733735 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:15.733769 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:15.733899 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:15.734090 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:19:15.734240 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":108,"threads_started":1}
I20260807 15:19:15.735898 30080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3: No bootstrap required, opened a new log
I20260807 15:19:15.738104 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:15.739738 30080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43305 } }
I20260807 15:19:15.740018 30080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:15.740085 30080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d209a11df3c4e36b5b6b89e434e3ad3, State: Initialized, Role: FOLLOWER
I20260807 15:19:15.740617 30080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43305 } }
I20260807 15:19:15.740784 30080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:15.740868 30080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:15.740975 30080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:15.742090 30080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43305 } }
I20260807 15:19:15.742468 30080 leader_election.cc:302] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d209a11df3c4e36b5b6b89e434e3ad3; no voters:
I20260807 15:19:15.742798 30080 leader_election.cc:288] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:15.742916 30096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:15.743185 30096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [term 1 LEADER]: Becoming Leader. State: Replica: 6d209a11df3c4e36b5b6b89e434e3ad3, State: Running, Role: LEADER
I20260807 15:19:15.743553 30096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43305 } }
I20260807 15:19:15.744167 30080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:15.745333 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:15.740269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:45301 (local address 127.0.0.1:43305)
0807 15:19:15.740384 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:15.740388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:15.740617 (+ 229us) server_negotiation.cc:407] Connection header received
0807 15:19:15.740696 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:15.740699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:15.740749 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:15.740830 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:15.741267 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:15.741871 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:15.743091 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:15.743448 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:15.743561 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:15.744508 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:15.744512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:15.744515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:15.744528 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:15.744555 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:15.744568 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:15.744664 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:15.745118 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 15:19:15.745159 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:19:15.745404 30098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d209a11df3c4e36b5b6b89e434e3ad3. Latest consensus state: current_term: 1 leader_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43305 } } }
I20260807 15:19:15.745568 30098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:15.746009 30100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d209a11df3c4e36b5b6b89e434e3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43305 } } }
I20260807 15:19:15.746177 30100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3 [sys.catalog]: This master's current role is: LEADER
W20260807 15:19:15.748847 30116 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:15.748914 30116 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:15.748994 30109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:15.749760 30109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:15.755216 30109 catalog_manager.cc:1347] Generated new cluster ID: 8f776a84a59240edb821ef8f0e4323ad
I20260807 15:19:15.755283 30109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:15.777673 30109 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:15.779026 30109 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:15.790867 30109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d209a11df3c4e36b5b6b89e434e3ad3: Generated new TSK 0
I20260807 15:19:15.791610 30109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:15.802277 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 29977
W20260807 15:19:15.816777 29953 connection.cc:535] client connection to 127.0.0.1:43305 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43305): Connection reset by peer (error 104)
2026-08-07T15:19:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:15 dist-test-slave-gkw7 krb5kdc[30127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:15 dist-test-slave-gkw7 krb5kdc[30127](info): set up 2 sockets
Aug 07 15:19:15 dist-test-slave-gkw7 krb5kdc[30127](info): commencing operation
krb5kdc: starting...
W20260807 15:19:17.052099 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:17 dist-test-slave-gkw7 krb5kdc[30127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:17.190225 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:17.308189 30148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:17.308531 30148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:17.308647 30148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:17.312817 30148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:17.312913 30148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:17.312951 30148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:17.312988 30148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:17.313019 30148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:17.318485 30148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:17.319607 30148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:17.320658 30148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:17.329560 30155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:17.329576 30158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:17.329670 30148 server_base.cc:1034] running on GCE node
W20260807 15:19:17.329581 30154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:17.330476 30148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:17.331658 30148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:17.332892 30148 hybrid_clock.cc:648] HybridClock initialized: now 1786115957332860 us; error 43 us; skew 500 ppm
Aug 07 15:19:17 dist-test-slave-gkw7 krb5kdc[30127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:17.335903 30148 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:17.336839 30148 webserver.cc:460] Webserver started at http://127.0.0.1:41065/ using document root <none> and password file <none>
I20260807 15:19:17.337422 30148 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:17.337519 30148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:17.337762 30148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:17.339890 30148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "99d0ddf792404989b009f905cead3554"
format_stamp: "Formatted at 2026-08-07 15:19:17 on dist-test-slave-gkw7"
server_key: "7fc41b739b062b647f6d814efd2450d3"
server_key_iv: "be8407cc7082600cdc83699a874ac794"
server_key_version: "encryptionkey@0"
I20260807 15:19:17.340483 30148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "99d0ddf792404989b009f905cead3554"
format_stamp: "Formatted at 2026-08-07 15:19:17 on dist-test-slave-gkw7"
server_key: "7fc41b739b062b647f6d814efd2450d3"
server_key_iv: "be8407cc7082600cdc83699a874ac794"
server_key_version: "encryptionkey@0"
I20260807 15:19:17.344359 30148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 15:19:17.346791 30175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:17.348083 30148 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:17.348228 30148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "99d0ddf792404989b009f905cead3554"
format_stamp: "Formatted at 2026-08-07 15:19:17 on dist-test-slave-gkw7"
server_key: "7fc41b739b062b647f6d814efd2450d3"
server_key_iv: "be8407cc7082600cdc83699a874ac794"
server_key_version: "encryptionkey@0"
I20260807 15:19:17.348340 30148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:17.352949 30148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:17.355794 30148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:17.355996 30148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:17.362963 30148 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37917
I20260807 15:19:17.363042 30256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37917 every 8 connection(s)
I20260807 15:19:17.363924 30148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:17.367074 30257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:17.367801 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 30148
I20260807 15:19:17.367957 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:17.368311 32667 external_mini_cluster.cc:1428] Setting key 55ee3159b12c014e5547ab64d70e7af9
I20260807 15:19:17.374504 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:17.369902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:53321 (local address 127.0.0.1:37917)
0807 15:19:17.370370 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:17.370394 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:17.370443 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:19:17.371333 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:17.371360 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:17.371767 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:17.372163 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:17.372951 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:17.372957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:17.372973 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:17.372987 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:17.373057 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:17.373088 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:17.373248 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:17.373328 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:19:17.373500 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":122,"threads_started":1}
I20260807 15:19:17.375504 30257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554: Bootstrap starting.
I20260807 15:19:17.377738 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:17.378253 30257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:17.379032 30257 log.cc:824] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:17.381409 30257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554: No bootstrap required, opened a new log
I20260807 15:19:17.382388 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:17.380253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:50369 (local address 127.0.0.1:37917)
0807 15:19:17.380390 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:17.380395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:17.380657 (+ 262us) server_negotiation.cc:407] Connection header received
0807 15:19:17.380772 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:17.380776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:17.380815 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:17.380911 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:17.381868 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:17.381873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:17.381878 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:17.381896 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:17.381928 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:17.381944 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:17.382045 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:17.382200 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 15:19:17.382245 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:19:17.385603 30257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0ddf792404989b009f905cead3554" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37917 } }
I20260807 15:19:17.385855 30257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:17.385964 30257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d0ddf792404989b009f905cead3554, State: Initialized, Role: FOLLOWER
I20260807 15:19:17.386423 30257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0ddf792404989b009f905cead3554" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37917 } }
I20260807 15:19:17.386555 30257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:17.386641 30257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:17.386768 30257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:17.387765 30257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0ddf792404989b009f905cead3554" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37917 } }
I20260807 15:19:17.388145 30257 leader_election.cc:302] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99d0ddf792404989b009f905cead3554; no voters:
I20260807 15:19:17.388531 30257 leader_election.cc:288] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:17.388741 30270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:17.389154 30270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [term 1 LEADER]: Becoming Leader. State: Replica: 99d0ddf792404989b009f905cead3554, State: Running, Role: LEADER
I20260807 15:19:17.389525 30270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0ddf792404989b009f905cead3554" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37917 } }
I20260807 15:19:17.390022 30257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:17.391673 30271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99d0ddf792404989b009f905cead3554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0ddf792404989b009f905cead3554" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37917 } } }
I20260807 15:19:17.391692 30273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99d0ddf792404989b009f905cead3554. Latest consensus state: current_term: 1 leader_uuid: "99d0ddf792404989b009f905cead3554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d0ddf792404989b009f905cead3554" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37917 } } }
I20260807 15:19:17.391820 30271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:17.391831 30273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:17.392241 30283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:17.395756 30283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:17.402531 30283 catalog_manager.cc:1347] Generated new cluster ID: 0f2a03fed1d845c8a230cc2d1e9efdc7
I20260807 15:19:17.402626 30283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:17.426772 30283 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:17.427208 30283 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:17.435509 30283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99d0ddf792404989b009f905cead3554: Generated new TSK 0
I20260807 15:19:17.436286 30283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:17.447398 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 30148
2026-08-07T15:19:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:17 dist-test-slave-gkw7 krb5kdc[30300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:17 dist-test-slave-gkw7 krb5kdc[30300](info): set up 2 sockets
Aug 07 15:19:17 dist-test-slave-gkw7 krb5kdc[30300](info): commencing operation
krb5kdc: starting...
W20260807 15:19:18.687482 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:18 dist-test-slave-gkw7 krb5kdc[30300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:18.820117 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:18.939803 30317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:18.940132 30317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:18.940222 30317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:18.944372 30317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:18.944476 30317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:18.944516 30317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:18.944576 30317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:18.944615 30317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:18.950284 30317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:18.951483 30317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:18.952591 30317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:18.961242 30331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:18.961242 30329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:18.961292 30327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:18.961630 30317 server_base.cc:1034] running on GCE node
I20260807 15:19:18.962221 30317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:18.963410 30317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:18.964607 30317 hybrid_clock.cc:648] HybridClock initialized: now 1786115958964591 us; error 30 us; skew 500 ppm
Aug 07 15:19:18 dist-test-slave-gkw7 krb5kdc[30300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:18.971732 30317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:18.972823 30317 webserver.cc:460] Webserver started at http://127.0.0.1:38929/ using document root <none> and password file <none>
I20260807 15:19:18.973474 30317 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:18.973596 30317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:18.973857 30317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:18.976192 30317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "06eb0ed8f3524ef0ab61f59fe310521a"
format_stamp: "Formatted at 2026-08-07 15:19:18 on dist-test-slave-gkw7"
server_key: "b650bc165c3a05d3b3c0e060f6b7ba30"
server_key_iv: "89bd8db7cf6af0d75273bb33909b647f"
server_key_version: "encryptionkey@0"
I20260807 15:19:18.976843 30317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "06eb0ed8f3524ef0ab61f59fe310521a"
format_stamp: "Formatted at 2026-08-07 15:19:18 on dist-test-slave-gkw7"
server_key: "b650bc165c3a05d3b3c0e060f6b7ba30"
server_key_iv: "89bd8db7cf6af0d75273bb33909b647f"
server_key_version: "encryptionkey@0"
I20260807 15:19:18.980790 30317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:19:18.983417 30345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:18.984802 30317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:19:18.984939 30317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "06eb0ed8f3524ef0ab61f59fe310521a"
format_stamp: "Formatted at 2026-08-07 15:19:18 on dist-test-slave-gkw7"
server_key: "b650bc165c3a05d3b3c0e060f6b7ba30"
server_key_iv: "89bd8db7cf6af0d75273bb33909b647f"
server_key_version: "encryptionkey@0"
I20260807 15:19:18.985054 30317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:19.007097 30317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:19.010607 30317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:19.010829 30317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:19.018808 30317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39029
I20260807 15:19:19.018848 30422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39029 every 8 connection(s)
I20260807 15:19:19.019831 30317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:19.023310 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:19.027978 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 30317
I20260807 15:19:19.028105 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:19.028432 32667 external_mini_cluster.cc:1428] Setting key 9c7a963c76102ff999eaca4adc9d901a
I20260807 15:19:19.030217 30423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a: Bootstrap starting.
I20260807 15:19:19.033452 30423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:19.034523 30423 log.cc:824] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:19.036952 30423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a: No bootstrap required, opened a new log
Aug 07 15:19:19 dist-test-slave-gkw7 krb5kdc[30300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:19:19.042299 30423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39029 } }
I20260807 15:19:19.042632 30423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:19.042701 30423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06eb0ed8f3524ef0ab61f59fe310521a, State: Initialized, Role: FOLLOWER
I20260807 15:19:19.043298 30423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39029 } }
I20260807 15:19:19.043488 30423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:19.043628 30423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:19.043807 30423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:19.045195 30423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39029 } }
I20260807 15:19:19.045629 30423 leader_election.cc:302] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06eb0ed8f3524ef0ab61f59fe310521a; no voters:
I20260807 15:19:19.046073 30423 leader_election.cc:288] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:19.046284 30430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:19.046482 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:19.030122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60377 (local address 127.0.0.1:39029)
0807 15:19:19.030657 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:19.030680 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:19.030728 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:19:19.031707 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:19.031736 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:19.032234 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:19.032659 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:19.033411 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:19.035125 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:19.036093 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:19.036857 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:19.037054 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:19.040267 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:19.040278 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:19.040305 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:19.040354 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:19.042654 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:19.043285 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:19.043292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:19.043298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:19.043463 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:19.043877 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:19.043881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:19.043883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:19.044449 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:19.044726 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:19.044988 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 15:19:19.045265 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":162,"threads_started":1}
I20260807 15:19:19.046639 30430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [term 1 LEADER]: Becoming Leader. State: Replica: 06eb0ed8f3524ef0ab61f59fe310521a, State: Running, Role: LEADER
I20260807 15:19:19.047111 30430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39029 } }
I20260807 15:19:19.047552 30423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:19.048959 30431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39029 } } }
I20260807 15:19:19.049180 30431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:19.049706 30445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:19.050720 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:19.050820 30433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06eb0ed8f3524ef0ab61f59fe310521a. Latest consensus state: current_term: 1 leader_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eb0ed8f3524ef0ab61f59fe310521a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39029 } } }
I20260807 15:19:19.050961 30433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:19.053483 30445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:19.060461 30445 catalog_manager.cc:1347] Generated new cluster ID: c0e2f9f898b444aaa00316f0f3350381
I20260807 15:19:19.060650 30445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:19.064253 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:19.053339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:36209 (local address 127.0.0.1:39029)
0807 15:19:19.053468 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:19.053474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:19.053775 (+ 301us) server_negotiation.cc:407] Connection header received
0807 15:19:19.053933 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:19.053937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:19.053995 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:19.054104 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:19.055075 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:19.055975 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:19.057267 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:19.057825 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:19.057949 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:19.059884 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:19.059896 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:19.059899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:19.059925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:19.062026 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:19.062698 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:19.062702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:19.062704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:19.062800 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:19.063212 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:19.063215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:19.063217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:19.063439 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:19.063580 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:19.063957 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 15:19:19.064074 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:19:19.071338 30445 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:19.072733 30445 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:19.084301 30445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06eb0ed8f3524ef0ab61f59fe310521a: Generated new TSK 0
I20260807 15:19:19.085095 30445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:19.096282 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 30317
W20260807 15:19:19.111315 30298 connection.cc:535] client connection to 127.0.0.1:39029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39029): Connection reset by peer (error 104)
2026-08-07T15:19:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1652 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:19 dist-test-slave-gkw7 krb5kdc[30471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:19 dist-test-slave-gkw7 krb5kdc[30471](info): set up 2 sockets
Aug 07 15:19:19 dist-test-slave-gkw7 krb5kdc[30471](info): commencing operation
krb5kdc: starting...
W20260807 15:19:20.330854 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:20 dist-test-slave-gkw7 krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:20.468804 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:20.583367 30493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:20.583688 30493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:20.583771 30493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:20.587815 30493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:20.587918 30493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:20.587956 30493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:20.587993 30493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:20.588029 30493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:20.593267 30493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:20.594416 30493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:20.595499 30493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:20.604261 30505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:20.604329 30507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:20.604267 30510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:20.604457 30493 server_base.cc:1034] running on GCE node
I20260807 15:19:20.605118 30493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:20.606366 30493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:20.607580 30493 hybrid_clock.cc:648] HybridClock initialized: now 1786115960607550 us; error 43 us; skew 500 ppm
Aug 07 15:19:20 dist-test-slave-gkw7 krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:20.610649 30493 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:20.611618 30493 webserver.cc:460] Webserver started at http://127.0.0.1:39259/ using document root <none> and password file <none>
I20260807 15:19:20.612174 30493 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:20.612257 30493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:20.612438 30493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:20.614646 30493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f"
format_stamp: "Formatted at 2026-08-07 15:19:20 on dist-test-slave-gkw7"
server_key: "b42d9f8f8eb4961c6630a7cc913db8c9"
server_key_iv: "c3f555921c5faef27326afec9e6ab49b"
server_key_version: "encryptionkey@0"
I20260807 15:19:20.615191 30493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f"
format_stamp: "Formatted at 2026-08-07 15:19:20 on dist-test-slave-gkw7"
server_key: "b42d9f8f8eb4961c6630a7cc913db8c9"
server_key_iv: "c3f555921c5faef27326afec9e6ab49b"
server_key_version: "encryptionkey@0"
I20260807 15:19:20.619154 30493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:19:20.621810 30519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:20.623073 30493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:19:20.623211 30493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f"
format_stamp: "Formatted at 2026-08-07 15:19:20 on dist-test-slave-gkw7"
server_key: "b42d9f8f8eb4961c6630a7cc913db8c9"
server_key_iv: "c3f555921c5faef27326afec9e6ab49b"
server_key_version: "encryptionkey@0"
I20260807 15:19:20.623322 30493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:20.637423 30493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:20.641134 30493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:20.641417 30493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:20.649211 30493 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43667
I20260807 15:19:20.649211 30589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43667 every 8 connection(s)
I20260807 15:19:20.650404 30493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:20.653417 30591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:20.656327 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 30493
I20260807 15:19:20.656450 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:20.656780 32667 external_mini_cluster.cc:1428] Setting key 9e07b5a5a49ebc364c1a8de6bb1792e3
I20260807 15:19:20.659974 30591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f: Bootstrap starting.
I20260807 15:19:20.662427 30591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:20.663223 30591 log.cc:824] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:20.665179 30591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f: No bootstrap required, opened a new log
I20260807 15:19:20.666251 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:20.658398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:46025 (local address 127.0.0.1:43667)
0807 15:19:20.658890 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:20.658905 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:20.658943 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:19:20.659591 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:20.659609 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:20.659983 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:20.660372 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:20.660791 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:20.661889 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:20.662875 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:20.663381 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:20.663561 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:20.664514 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:20.664519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:20.664533 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:20.664926 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:20.664984 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:20.665006 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:20.665153 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:20.665313 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:19:20.665460 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":150,"threads_started":1}
I20260807 15:19:20.669454 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:20.669608 30591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 15:19:20.669879 30591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:20.669960 30591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa2fe14d9fb4bfabb565f47310a1e2f, State: Initialized, Role: FOLLOWER
I20260807 15:19:20.670367 30591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 15:19:20.670518 30591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:20.670606 30591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:20.670713 30591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:20.671625 30591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 15:19:20.672025 30591 leader_election.cc:302] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fa2fe14d9fb4bfabb565f47310a1e2f; no voters:
I20260807 15:19:20.672402 30591 leader_election.cc:288] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:20.672508 30606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:20.672757 30606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [term 1 LEADER]: Becoming Leader. State: Replica: 2fa2fe14d9fb4bfabb565f47310a1e2f, State: Running, Role: LEADER
I20260807 15:19:20.673133 30606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 15:19:20.673776 30591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:20.675261 30607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } } }
I20260807 15:19:20.675392 30607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:20.676091 30609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fa2fe14d9fb4bfabb565f47310a1e2f. Latest consensus state: current_term: 1 leader_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa2fe14d9fb4bfabb565f47310a1e2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } } }
I20260807 15:19:20.676805 30609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:20.677155 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:20.671790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:47943 (local address 127.0.0.1:43667)
0807 15:19:20.671920 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:20.671925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:20.672139 (+ 214us) server_negotiation.cc:407] Connection header received
0807 15:19:20.672388 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:20.672393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:20.672449 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:20.672539 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:20.672985 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:20.673764 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:20.674916 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:20.675333 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:20.675434 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:20.676576 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:20.676580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:20.676583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:20.676596 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:20.676623 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:20.676637 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:20.676726 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:20.676993 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 15:19:20.677029 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:19:20.679523 30633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:20.679589 30633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:20.679713 30627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:20.680452 30627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:20.686435 30627 catalog_manager.cc:1347] Generated new cluster ID: 23ba97d8c8954bc38becf2bcc10e1176
I20260807 15:19:20.686515 30627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:20.695758 30627 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:20.697039 30627 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:20.704566 30627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fa2fe14d9fb4bfabb565f47310a1e2f: Generated new TSK 0
I20260807 15:19:20.705312 30627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:20.715521 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 30493
W20260807 15:19:20.729029 30469 connection.cc:535] client connection to 127.0.0.1:43667 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43667): Connection reset by peer (error 104)
2026-08-07T15:19:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1617 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:20 dist-test-slave-gkw7 krb5kdc[30641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:20 dist-test-slave-gkw7 krb5kdc[30641](info): set up 2 sockets
Aug 07 15:19:20 dist-test-slave-gkw7 krb5kdc[30641](info): commencing operation
krb5kdc: starting...
W20260807 15:19:22.021092 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:22 dist-test-slave-gkw7 krb5kdc[30641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:19:22.156436 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38679
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:22.270936 30663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:22.271252 30663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:22.271296 30663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:22.275652 30663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:22.275730 30663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:22.275750 30663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:22.275766 30663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:22.275782 30663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:22.281338 30663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38679
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:22.282579 30663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:22.283689 30663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:22.292519 30674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:22.292544 30669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:22.292536 30671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:22.292696 30663 server_base.cc:1034] running on GCE node
I20260807 15:19:22.293421 30663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:22.294561 30663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:22.295765 30663 hybrid_clock.cc:648] HybridClock initialized: now 1786115962295752 us; error 30 us; skew 500 ppm
Aug 07 15:19:22 dist-test-slave-gkw7 krb5kdc[30641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:22.298775 30663 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:19:22.299721 30663 webserver.cc:460] Webserver started at http://127.0.0.1:32921/ using document root <none> and password file <none>
I20260807 15:19:22.300266 30663 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:22.300328 30663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:22.300498 30663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:22.302644 30663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "4787301f6ec84d42841f8e8dd4937dab"
format_stamp: "Formatted at 2026-08-07 15:19:22 on dist-test-slave-gkw7"
server_key: "efa94a46e3460aa3326ac908b3a2c4fc"
server_key_iv: "3fdfb362c60dd39f1d1955092af9e45b"
server_key_version: "encryptionkey@0"
I20260807 15:19:22.303188 30663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "4787301f6ec84d42841f8e8dd4937dab"
format_stamp: "Formatted at 2026-08-07 15:19:22 on dist-test-slave-gkw7"
server_key: "efa94a46e3460aa3326ac908b3a2c4fc"
server_key_iv: "3fdfb362c60dd39f1d1955092af9e45b"
server_key_version: "encryptionkey@0"
I20260807 15:19:22.307111 30663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:19:22.309973 30681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:22.311491 30663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:19:22.311643 30663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "4787301f6ec84d42841f8e8dd4937dab"
format_stamp: "Formatted at 2026-08-07 15:19:22 on dist-test-slave-gkw7"
server_key: "efa94a46e3460aa3326ac908b3a2c4fc"
server_key_iv: "3fdfb362c60dd39f1d1955092af9e45b"
server_key_version: "encryptionkey@0"
I20260807 15:19:22.311766 30663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:22.316674 30663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:22.319818 30663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:22.320021 30663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:22.327977 30663 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38679
I20260807 15:19:22.328007 30776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38679 every 8 connection(s)
I20260807 15:19:22.328981 30663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:22.331820 30777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:22.334105 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 30663
I20260807 15:19:22.334260 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:22.334564 32667 external_mini_cluster.cc:1428] Setting key c583606cc96c20891840e3229988eed6
I20260807 15:19:22.338682 30777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab: Bootstrap starting.
I20260807 15:19:22.340512 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:22.336077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:44693 (local address 127.0.0.1:38679)
0807 15:19:22.336558 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:22.336580 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:22.336625 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:19:22.337515 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:22.337561 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:22.337943 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:22.338312 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:22.339087 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:22.339093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:22.339107 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:22.339119 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:22.339182 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:22.339208 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:22.339342 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:22.339409 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:19:22.339568 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":142,"threads_started":1}
I20260807 15:19:22.341480 30777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:22.342271 30777 log.cc:824] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:22.343751 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:22.344312 30777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab: No bootstrap required, opened a new log
I20260807 15:19:22.347805 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:22.346076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:51003 (local address 127.0.0.1:38679)
0807 15:19:22.346207 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:22.346211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:22.346334 (+ 123us) server_negotiation.cc:407] Connection header received
0807 15:19:22.346432 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:22.346434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:22.346464 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:22.346534 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:22.347359 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:22.347362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:22.347364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:22.347374 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:22.347400 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:22.347411 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:22.347484 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:22.347623 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:19:22.347661 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:19:22.348862 30777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4787301f6ec84d42841f8e8dd4937dab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38679 } }
I20260807 15:19:22.349151 30777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:22.349285 30777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4787301f6ec84d42841f8e8dd4937dab, State: Initialized, Role: FOLLOWER
I20260807 15:19:22.349767 30777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4787301f6ec84d42841f8e8dd4937dab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38679 } }
I20260807 15:19:22.349924 30777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:22.349992 30777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:22.350090 30777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:22.351018 30777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4787301f6ec84d42841f8e8dd4937dab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38679 } }
I20260807 15:19:22.351399 30777 leader_election.cc:302] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4787301f6ec84d42841f8e8dd4937dab; no voters:
I20260807 15:19:22.351790 30777 leader_election.cc:288] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:22.351948 30793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:22.352203 30793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [term 1 LEADER]: Becoming Leader. State: Replica: 4787301f6ec84d42841f8e8dd4937dab, State: Running, Role: LEADER
I20260807 15:19:22.352519 30793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4787301f6ec84d42841f8e8dd4937dab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38679 } }
I20260807 15:19:22.353106 30777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:22.354329 30794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4787301f6ec84d42841f8e8dd4937dab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4787301f6ec84d42841f8e8dd4937dab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38679 } } }
I20260807 15:19:22.354457 30794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:22.354763 30795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4787301f6ec84d42841f8e8dd4937dab. Latest consensus state: current_term: 1 leader_uuid: "4787301f6ec84d42841f8e8dd4937dab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4787301f6ec84d42841f8e8dd4937dab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38679 } } }
I20260807 15:19:22.354856 30795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:22.354879 30805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:22.358049 30805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:22.364363 30805 catalog_manager.cc:1347] Generated new cluster ID: 4b0da433c1584d68916c9b988c8c5c14
I20260807 15:19:22.364452 30805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:22.376252 30805 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:22.376641 30805 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:22.381446 30805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4787301f6ec84d42841f8e8dd4937dab: Generated new TSK 0
I20260807 15:19:22.382238 30805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:22.392695 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 30663
2026-08-07T15:19:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1679 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:22 dist-test-slave-gkw7 krb5kdc[30822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:22 dist-test-slave-gkw7 krb5kdc[30822](info): set up 2 sockets
Aug 07 15:19:22 dist-test-slave-gkw7 krb5kdc[30822](info): commencing operation
krb5kdc: starting...
W20260807 15:19:23.652379 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:23 dist-test-slave-gkw7 krb5kdc[30822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:23.784773 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:41605
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:41605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:23.901297 30838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:23.901655 30838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:23.901739 30838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:23.905761 30838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:23.905859 30838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:23.905900 30838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:23.905957 30838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:23.905997 30838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:23.911314 30838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:41605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:41605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:23.912487 30838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:23.913645 30838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:19:23.922881 30838 server_base.cc:1034] running on GCE node
W20260807 15:19:23.922782 30849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:23.922782 30853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:23.922832 30846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:23.923884 30838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:23.924938 30838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:23.926096 30838 hybrid_clock.cc:648] HybridClock initialized: now 1786115963926064 us; error 50 us; skew 500 ppm
Aug 07 15:19:23 dist-test-slave-gkw7 krb5kdc[30822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115963, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:23.928891 30838 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:23.929900 30838 webserver.cc:460] Webserver started at http://127.31.230.254:38911/ using document root <none> and password file <none>
I20260807 15:19:23.930482 30838 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:23.930576 30838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:23.930832 30838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:23.933013 30838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "41030eab1cc84727b8bf6039b76538f6"
format_stamp: "Formatted at 2026-08-07 15:19:23 on dist-test-slave-gkw7"
server_key: "afd7a511cec2fdfed35648cbb05ddf83"
server_key_iv: "cc6c666b5f98388dc028592825d935bd"
server_key_version: "encryptionkey@0"
I20260807 15:19:23.933676 30838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "41030eab1cc84727b8bf6039b76538f6"
format_stamp: "Formatted at 2026-08-07 15:19:23 on dist-test-slave-gkw7"
server_key: "afd7a511cec2fdfed35648cbb05ddf83"
server_key_iv: "cc6c666b5f98388dc028592825d935bd"
server_key_version: "encryptionkey@0"
I20260807 15:19:23.937481 30838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:19:23.940037 30863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:23.941501 30838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:19:23.941742 30838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "41030eab1cc84727b8bf6039b76538f6"
format_stamp: "Formatted at 2026-08-07 15:19:23 on dist-test-slave-gkw7"
server_key: "afd7a511cec2fdfed35648cbb05ddf83"
server_key_iv: "cc6c666b5f98388dc028592825d935bd"
server_key_version: "encryptionkey@0"
I20260807 15:19:23.941884 30838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:23.994431 30838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:23.998147 30838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:23.998377 30838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:24.005797 30838 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:41605
I20260807 15:19:24.005893 30942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:41605 every 8 connection(s)
I20260807 15:19:24.006884 30838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:24.009848 30943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:24.012578 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 30838
I20260807 15:19:24.012746 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:24.013055 32667 external_mini_cluster.cc:1428] Setting key 85fd8f3be4e8d7d4f97c62e19a77f5a9
I20260807 15:19:24.016069 30943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6: Bootstrap starting.
I20260807 15:19:24.018819 30943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:24.019727 30943 log.cc:824] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:24.021952 30943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6: No bootstrap required, opened a new log
Aug 07 15:19:24 dist-test-slave-gkw7 krb5kdc[30822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:24.026563 30943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41030eab1cc84727b8bf6039b76538f6" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41605 } }
I20260807 15:19:24.026877 30943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:24.026985 30943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41030eab1cc84727b8bf6039b76538f6, State: Initialized, Role: FOLLOWER
I20260807 15:19:24.027624 30943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41030eab1cc84727b8bf6039b76538f6" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41605 } }
I20260807 15:19:24.027874 30943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:24.027966 30943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:24.028087 30943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:24.029454 30943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41030eab1cc84727b8bf6039b76538f6" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41605 } }
I20260807 15:19:24.029706 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:24.014730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54947 (local address 127.31.230.254:41605)
0807 15:19:24.015169 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:24.015186 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:24.015230 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:19:24.015869 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:24.015887 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:24.016280 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:24.016576 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:24.017571 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:24.018593 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:24.019849 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:24.020346 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:24.020483 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:24.023549 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:24.023554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:24.023567 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:24.023594 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:24.025474 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:24.026174 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:24.026179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:24.026183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:24.026296 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:24.026707 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:24.026709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:24.026710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:24.027263 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:24.027492 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:24.028242 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 15:19:24.028500 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":143,"threads_started":1}
I20260807 15:19:24.029860 30943 leader_election.cc:302] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41030eab1cc84727b8bf6039b76538f6; no voters:
I20260807 15:19:24.030200 30943 leader_election.cc:288] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:24.030318 30951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:24.030545 30951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [term 1 LEADER]: Becoming Leader. State: Replica: 41030eab1cc84727b8bf6039b76538f6, State: Running, Role: LEADER
I20260807 15:19:24.030915 30951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41030eab1cc84727b8bf6039b76538f6" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41605 } }
I20260807 15:19:24.031518 30943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:24.032719 30953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41030eab1cc84727b8bf6039b76538f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41030eab1cc84727b8bf6039b76538f6" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41605 } } }
I20260807 15:19:24.032931 30953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:24.033166 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:24.034055 30954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41030eab1cc84727b8bf6039b76538f6. Latest consensus state: current_term: 1 leader_uuid: "41030eab1cc84727b8bf6039b76538f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41030eab1cc84727b8bf6039b76538f6" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41605 } } }
I20260807 15:19:24.034438 30954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6 [sys.catalog]: This master's current role is: LEADER
W20260807 15:19:24.037271 30975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:24.037364 30975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:24.037468 30966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:24.038386 30966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:24.044562 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:24.035480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55759 (local address 127.31.230.254:41605)
0807 15:19:24.035610 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:24.035615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:24.035790 (+ 175us) server_negotiation.cc:407] Connection header received
0807 15:19:24.035895 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:24.035898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:24.035944 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:24.036024 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:24.036855 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:24.037692 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:24.038837 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:24.039203 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:24.039315 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:24.040910 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:24.040913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:24.040916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:24.040947 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:24.042577 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:24.043148 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:24.043152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:24.043154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:24.043254 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:24.043644 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:24.043647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:24.043649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:24.043832 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:24.043992 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:24.044279 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 15:19:24.044377 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:19:24.045113 30966 catalog_manager.cc:1347] Generated new cluster ID: a5f4b14e5c974d158bf593e43eef01b0
I20260807 15:19:24.045176 30966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:24.075297 30966 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:24.076340 30966 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:24.083639 30966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41030eab1cc84727b8bf6039b76538f6: Generated new TSK 0
I20260807 15:19:24.084342 30966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:24.092929 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 30838
W20260807 15:19:24.107429 30819 connection.cc:535] client connection to 127.31.230.254:41605 recv error: Network error: failed to read from TLS socket (remote: 127.31.230.254:41605): Connection reset by peer (error 104)
2026-08-07T15:19:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1699 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:24 dist-test-slave-gkw7 krb5kdc[30992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:24 dist-test-slave-gkw7 krb5kdc[30992](info): set up 2 sockets
Aug 07 15:19:24 dist-test-slave-gkw7 krb5kdc[30992](info): commencing operation
krb5kdc: starting...
W20260807 15:19:25.316035 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:25 dist-test-slave-gkw7 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:25.456337 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38465
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:38465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:25.572805 31011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:25.573078 31011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:25.573122 31011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:25.577131 31011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:25.577186 31011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:25.577205 31011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:25.577222 31011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:25.577266 31011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:25.582453 31011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:38465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:25.583478 31011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:25.584510 31011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:25.593135 31025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:25.593145 31024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:25.593135 31028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:25.593238 31011 server_base.cc:1034] running on GCE node
I20260807 15:19:25.594002 31011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:25.595086 31011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:25.596283 31011 hybrid_clock.cc:648] HybridClock initialized: now 1786115965596255 us; error 43 us; skew 500 ppm
Aug 07 15:19:25 dist-test-slave-gkw7 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115965, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:25.599073 31011 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:25.599996 31011 webserver.cc:460] Webserver started at http://127.31.230.254:35945/ using document root <none> and password file <none>
I20260807 15:19:25.600570 31011 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:25.600665 31011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:25.600903 31011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:25.603000 31011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68"
format_stamp: "Formatted at 2026-08-07 15:19:25 on dist-test-slave-gkw7"
server_key: "3804e5c2e92110f7cd313b709942dd5f"
server_key_iv: "22d198c10f97eac1bf7f10f1abcaf647"
server_key_version: "encryptionkey@0"
I20260807 15:19:25.603586 31011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68"
format_stamp: "Formatted at 2026-08-07 15:19:25 on dist-test-slave-gkw7"
server_key: "3804e5c2e92110f7cd313b709942dd5f"
server_key_iv: "22d198c10f97eac1bf7f10f1abcaf647"
server_key_version: "encryptionkey@0"
I20260807 15:19:25.607321 31011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 15:19:25.609817 31038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:25.611040 31011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:19:25.611186 31011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68"
format_stamp: "Formatted at 2026-08-07 15:19:25 on dist-test-slave-gkw7"
server_key: "3804e5c2e92110f7cd313b709942dd5f"
server_key_iv: "22d198c10f97eac1bf7f10f1abcaf647"
server_key_version: "encryptionkey@0"
I20260807 15:19:25.611321 31011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:25.639452 31011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:25.644117 31011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:25.644361 31011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:25.655171 31011 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:38465
I20260807 15:19:25.655206 31118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:38465 every 8 connection(s)
I20260807 15:19:25.656332 31011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:25.660447 31120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:25.664737 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31011
I20260807 15:19:25.664925 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:25.665297 32667 external_mini_cluster.cc:1428] Setting key 122ecfe8c30b3adde71b115ab368f775
I20260807 15:19:25.669224 31120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68: Bootstrap starting.
I20260807 15:19:25.672592 31120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:25.673722 31120 log.cc:824] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:25.675773 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:25.666811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45267 (local address 127.31.230.254:38465)
0807 15:19:25.667333 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:25.667355 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:25.667404 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:19:25.668055 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:25.668072 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:25.668419 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:25.668788 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:25.669219 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:25.670384 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:25.671768 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:25.672268 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:25.672459 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:25.673673 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:25.673680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:25.673723 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:25.674219 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:25.674296 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:25.674331 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:25.674499 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:25.674701 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:19:25.675042 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":181,"threads_started":1}
I20260807 15:19:25.676152 31120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68: No bootstrap required, opened a new log
I20260807 15:19:25.678884 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:25.680420 31120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38465 } }
I20260807 15:19:25.680636 31120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:25.680693 31120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4c1bdeeb0a4f91b6cd443b7cb14e68, State: Initialized, Role: FOLLOWER
I20260807 15:19:25.681209 31120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38465 } }
I20260807 15:19:25.681399 31120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:25.681512 31120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:25.681757 31120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:25.682931 31120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38465 } }
I20260807 15:19:25.683300 31120 leader_election.cc:302] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c4c1bdeeb0a4f91b6cd443b7cb14e68; no voters:
I20260807 15:19:25.683595 31120 leader_election.cc:288] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:25.683718 31137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:25.683979 31137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [term 1 LEADER]: Becoming Leader. State: Replica: 2c4c1bdeeb0a4f91b6cd443b7cb14e68, State: Running, Role: LEADER
I20260807 15:19:25.684325 31137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38465 } }
I20260807 15:19:25.684914 31120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:25.686024 31139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38465 } } }
I20260807 15:19:25.686168 31139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:25.686595 31150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:25.686560 31140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4c1bdeeb0a4f91b6cd443b7cb14e68. Latest consensus state: current_term: 1 leader_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4c1bdeeb0a4f91b6cd443b7cb14e68" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38465 } } }
I20260807 15:19:25.687214 31140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:25.687953 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:25.681049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35335 (local address 127.31.230.254:38465)
0807 15:19:25.681147 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:25.681151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:25.681311 (+ 160us) server_negotiation.cc:407] Connection header received
0807 15:19:25.681416 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:25.681419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:25.681470 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:25.681605 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:25.681973 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:25.682778 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:25.683759 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:25.684088 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:25.684187 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:25.687393 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:25.687398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:25.687401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:25.687414 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:25.687443 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:25.687457 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:25.687546 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:25.687771 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 15:19:25.687805 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:19:25.689872 31150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:25.695865 31150 catalog_manager.cc:1347] Generated new cluster ID: 7f5e135a6a4c4356adead0a365d72c01
I20260807 15:19:25.695943 31150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:25.712455 31150 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:25.713492 31150 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:25.718884 31150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c4c1bdeeb0a4f91b6cd443b7cb14e68: Generated new TSK 0
I20260807 15:19:25.719543 31150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:25.727336 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31011
W20260807 15:19:25.742375 30989 connection.cc:535] client connection to 127.31.230.254:38465 recv error: Network error: failed to read from TLS socket (remote: 127.31.230.254:38465): Connection reset by peer (error 104)
2026-08-07T15:19:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:25 dist-test-slave-gkw7 krb5kdc[31166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:25 dist-test-slave-gkw7 krb5kdc[31166](info): set up 2 sockets
Aug 07 15:19:25 dist-test-slave-gkw7 krb5kdc[31166](info): commencing operation
krb5kdc: starting...
W20260807 15:19:26.950668 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:27 dist-test-slave-gkw7 krb5kdc[31166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:27.086122 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37381
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:37381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:27.202821 31186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:27.203143 31186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:27.203233 31186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:27.207250 31186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:27.207352 31186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:27.207391 31186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:27.207428 31186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:27.207460 31186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:27.212695 31186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:37381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:37381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:27.213788 31186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:27.214828 31186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:27.223856 31192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:27.223860 31199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:27.223860 31191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:27.224258 31186 server_base.cc:1034] running on GCE node
I20260807 15:19:27.224817 31186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:27.226042 31186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:27.227244 31186 hybrid_clock.cc:648] HybridClock initialized: now 1786115967227230 us; error 26 us; skew 500 ppm
Aug 07 15:19:27 dist-test-slave-gkw7 krb5kdc[31166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:27.230177 31186 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:27.231014 31186 webserver.cc:460] Webserver started at http://127.31.230.254:42815/ using document root <none> and password file <none>
I20260807 15:19:27.231535 31186 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:27.231601 31186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:27.231771 31186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:27.233904 31186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "2ca7b3fcd0d54577aa2c5fdabe940480"
format_stamp: "Formatted at 2026-08-07 15:19:27 on dist-test-slave-gkw7"
server_key: "331aeb09f93ed2e9d0210628d7db6d3b"
server_key_iv: "435e0829a3ee56e48c4ef194c4154555"
server_key_version: "encryptionkey@0"
I20260807 15:19:27.234431 31186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "2ca7b3fcd0d54577aa2c5fdabe940480"
format_stamp: "Formatted at 2026-08-07 15:19:27 on dist-test-slave-gkw7"
server_key: "331aeb09f93ed2e9d0210628d7db6d3b"
server_key_iv: "435e0829a3ee56e48c4ef194c4154555"
server_key_version: "encryptionkey@0"
I20260807 15:19:27.238164 31186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:19:27.240656 31216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:27.241986 31186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:19:27.242090 31186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "2ca7b3fcd0d54577aa2c5fdabe940480"
format_stamp: "Formatted at 2026-08-07 15:19:27 on dist-test-slave-gkw7"
server_key: "331aeb09f93ed2e9d0210628d7db6d3b"
server_key_iv: "435e0829a3ee56e48c4ef194c4154555"
server_key_version: "encryptionkey@0"
I20260807 15:19:27.242177 31186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:27.246629 31186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:27.249951 31186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:27.250099 31186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:27.257189 31186 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:37381
I20260807 15:19:27.257220 31289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:37381 every 8 connection(s)
I20260807 15:19:27.258162 31186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:27.261247 31291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:27.263784 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31186
I20260807 15:19:27.263936 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:27.264253 32667 external_mini_cluster.cc:1428] Setting key 1930c123d314f8c3fa0b2c02fdf14711
I20260807 15:19:27.267560 31291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480: Bootstrap starting.
I20260807 15:19:27.269670 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:27.265738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38395 (local address 127.31.230.254:37381)
0807 15:19:27.266248 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:27.266264 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:27.266302 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:19:27.266944 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:27.266961 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:27.267255 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:27.267598 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:27.268413 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:27.268418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:27.268430 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:27.268440 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:27.268493 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:27.268512 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:27.268689 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:27.268760 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:19:27.268919 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":143,"threads_started":1}
I20260807 15:19:27.270174 31291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:27.271036 31291 log.cc:824] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:27.272593 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:27.273403 31291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480: No bootstrap required, opened a new log
I20260807 15:19:27.278050 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:27.275408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59873 (local address 127.31.230.254:37381)
0807 15:19:27.275574 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:27.275578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:27.275866 (+ 288us) server_negotiation.cc:407] Connection header received
0807 15:19:27.276026 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:27.276029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:27.276068 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:27.276160 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:27.277521 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:27.277526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:27.277550 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:27.277566 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:27.277601 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:27.277617 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:27.277723 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:27.277881 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 15:19:27.277921 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 15:19:27.278996 31291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37381 } }
I20260807 15:19:27.279286 31291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:27.279346 31291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca7b3fcd0d54577aa2c5fdabe940480, State: Initialized, Role: FOLLOWER
I20260807 15:19:27.279819 31291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37381 } }
I20260807 15:19:27.279947 31291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:27.280027 31291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:27.280141 31291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:27.281551 31291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37381 } }
I20260807 15:19:27.281965 31291 leader_election.cc:302] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ca7b3fcd0d54577aa2c5fdabe940480; no voters:
I20260807 15:19:27.282337 31291 leader_election.cc:288] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:27.282493 31306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:27.282757 31306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [term 1 LEADER]: Becoming Leader. State: Replica: 2ca7b3fcd0d54577aa2c5fdabe940480, State: Running, Role: LEADER
I20260807 15:19:27.283059 31306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37381 } }
I20260807 15:19:27.283725 31291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:27.284888 31308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ca7b3fcd0d54577aa2c5fdabe940480. Latest consensus state: current_term: 1 leader_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37381 } } }
I20260807 15:19:27.285022 31308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:27.285449 31317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:27.285375 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca7b3fcd0d54577aa2c5fdabe940480" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 37381 } } }
I20260807 15:19:27.285620 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:27.288849 31317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:27.294747 31317 catalog_manager.cc:1347] Generated new cluster ID: 9ffa38913d5f4d47847902066cb23dae
I20260807 15:19:27.294818 31317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:27.305933 31317 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:27.306279 31317 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:27.316365 31317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ca7b3fcd0d54577aa2c5fdabe940480: Generated new TSK 0
I20260807 15:19:27.317010 31317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:27.322084 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31186
2026-08-07T15:19:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:27 dist-test-slave-gkw7 krb5kdc[31336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:27 dist-test-slave-gkw7 krb5kdc[31336](info): set up 2 sockets
Aug 07 15:19:27 dist-test-slave-gkw7 krb5kdc[31336](info): commencing operation
krb5kdc: starting...
W20260807 15:19:28.628726 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:28 dist-test-slave-gkw7 krb5kdc[31336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:28.760612 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39431
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:39431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:28.872961 31351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:28.873251 31351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:28.873333 31351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:28.877364 31351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:28.877455 31351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:28.877496 31351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:28.877559 31351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:28.877595 31351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:28.882774 31351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:39431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:39431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:28.883845 31351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:28.884862 31351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:28.893579 31359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:28.893617 31363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:28.893617 31358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:28.893810 31351 server_base.cc:1034] running on GCE node
I20260807 15:19:28.894412 31351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:28.895515 31351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:28.896708 31351 hybrid_clock.cc:648] HybridClock initialized: now 1786115968896693 us; error 35 us; skew 500 ppm
Aug 07 15:19:28 dist-test-slave-gkw7 krb5kdc[31336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:28.899533 31351 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:28.900398 31351 webserver.cc:460] Webserver started at http://127.31.230.254:41421/ using document root <none> and password file <none>
I20260807 15:19:28.900903 31351 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:28.900961 31351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:28.901136 31351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:28.903363 31351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "2f01419b438144558f9aca20c35d2d6d"
format_stamp: "Formatted at 2026-08-07 15:19:28 on dist-test-slave-gkw7"
server_key: "4ae6ef388738ea48ed35715cf0c6e100"
server_key_iv: "319282c191f4e185544adc1526f728a8"
server_key_version: "encryptionkey@0"
I20260807 15:19:28.903874 31351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "2f01419b438144558f9aca20c35d2d6d"
format_stamp: "Formatted at 2026-08-07 15:19:28 on dist-test-slave-gkw7"
server_key: "4ae6ef388738ea48ed35715cf0c6e100"
server_key_iv: "319282c191f4e185544adc1526f728a8"
server_key_version: "encryptionkey@0"
I20260807 15:19:28.908701 31351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 15:19:28.911185 31371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:28.912425 31351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:28.912539 31351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "2f01419b438144558f9aca20c35d2d6d"
format_stamp: "Formatted at 2026-08-07 15:19:28 on dist-test-slave-gkw7"
server_key: "4ae6ef388738ea48ed35715cf0c6e100"
server_key_iv: "319282c191f4e185544adc1526f728a8"
server_key_version: "encryptionkey@0"
I20260807 15:19:28.912624 31351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:28.926208 31351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:28.929476 31351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:28.929702 31351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:28.936949 31351 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:39431
I20260807 15:19:28.936967 31452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:39431 every 8 connection(s)
I20260807 15:19:28.937985 31351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:28.940795 31453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:28.946606 31453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d: Bootstrap starting.
I20260807 15:19:28.947119 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31351
I20260807 15:19:28.947294 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:28.947619 32667 external_mini_cluster.cc:1428] Setting key 60ccc512ad12c062c71f5b76daeccb2a
I20260807 15:19:28.949455 31453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:28.950233 31453 log.cc:824] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:28.952118 31453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d: No bootstrap required, opened a new log
I20260807 15:19:28.955515 31453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f01419b438144558f9aca20c35d2d6d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39431 } }
I20260807 15:19:28.955775 31453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:28.955855 31453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f01419b438144558f9aca20c35d2d6d, State: Initialized, Role: FOLLOWER
I20260807 15:19:28.956344 31453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f01419b438144558f9aca20c35d2d6d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39431 } }
I20260807 15:19:28.956511 31453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:28.956619 31453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:28.956746 31453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 0 FOLLOWER]: Advancing to term 1
Aug 07 15:19:28 dist-test-slave-gkw7 krb5kdc[31336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:28.957950 31453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f01419b438144558f9aca20c35d2d6d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39431 } }
I20260807 15:19:28.958313 31453 leader_election.cc:302] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f01419b438144558f9aca20c35d2d6d; no voters:
I20260807 15:19:28.958676 31453 leader_election.cc:288] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:28.958914 31459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:28.959405 31459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [term 1 LEADER]: Becoming Leader. State: Replica: 2f01419b438144558f9aca20c35d2d6d, State: Running, Role: LEADER
I20260807 15:19:28.959765 31459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f01419b438144558f9aca20c35d2d6d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39431 } }
I20260807 15:19:28.960098 31453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:28.961061 31459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f01419b438144558f9aca20c35d2d6d. Latest consensus state: current_term: 1 leader_uuid: "2f01419b438144558f9aca20c35d2d6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f01419b438144558f9aca20c35d2d6d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39431 } } }
I20260807 15:19:28.961197 31459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:28.961548 31460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f01419b438144558f9aca20c35d2d6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f01419b438144558f9aca20c35d2d6d" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 39431 } } }
I20260807 15:19:28.961655 31460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:28.961866 31471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:28.964298 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:28.948902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59957 (local address 127.31.230.254:39431)
0807 15:19:28.949420 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:28.949447 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:28.949496 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:19:28.950334 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:28.950378 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:28.950774 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:28.951199 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:28.952072 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:28.953249 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:28.954299 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:28.954826 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:28.955015 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:28.958148 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:28.958153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:28.958167 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:28.958195 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:28.960360 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:28.960910 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:28.960915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:28.960921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:28.961074 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:28.961460 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:28.961462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:28.961463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:28.962061 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:28.962876 (+ 815us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:28.963030 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 15:19:28.963357 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":145,"threads_started":1}
I20260807 15:19:28.965167 31471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:28.967432 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:28.975188 31471 catalog_manager.cc:1347] Generated new cluster ID: 10eb734f3b0142b49adc364ff44c000b
I20260807 15:19:28.975261 31471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:28.980532 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:28.970174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53641 (local address 127.31.230.254:39431)
0807 15:19:28.970297 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:28.970302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:28.970316 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:19:28.970355 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:28.970358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:28.970403 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:28.970491 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:28.971158 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:28.971956 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:28.972790 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:28.973218 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:28.973357 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:28.974726 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:28.974729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:28.974732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:28.974782 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:28.978797 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:28.979306 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:28.979310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:28.979312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:28.979396 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:28.979691 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:28.979694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:28.979696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:28.979889 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:28.980000 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:28.980273 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 15:19:28.980386 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:19:28.984421 31471 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:28.985320 31471 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:28.995419 31471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f01419b438144558f9aca20c35d2d6d: Generated new TSK 0
I20260807 15:19:28.996201 31471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:29.006826 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31351
2026-08-07T15:19:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1682 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:29 dist-test-slave-gkw7 krb5kdc[31493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:29 dist-test-slave-gkw7 krb5kdc[31493](info): set up 2 sockets
Aug 07 15:19:29 dist-test-slave-gkw7 krb5kdc[31493](info): commencing operation
krb5kdc: starting...
W20260807 15:19:30.241443 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:30 dist-test-slave-gkw7 krb5kdc[31493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:30.375013 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:41005
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:41005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:30.504150 31511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:30.504498 31511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:30.504590 31511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:30.508863 31511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:30.508977 31511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:30.509032 31511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:30.509097 31511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:30.509152 31511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:30.514674 31511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:41005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:41005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:30.515790 31511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:30.516885 31511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:30.526378 31518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:30.526428 31511 server_base.cc:1034] running on GCE node
W20260807 15:19:30.526386 31516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:30.526516 31521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:30.527155 31511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:30.528311 31511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:30.529578 31511 hybrid_clock.cc:648] HybridClock initialized: now 1786115970529491 us; error 48 us; skew 500 ppm
Aug 07 15:19:30 dist-test-slave-gkw7 krb5kdc[31493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115970, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:30.532634 31511 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:30.533746 31511 webserver.cc:460] Webserver started at http://127.31.230.254:34291/ using document root <none> and password file <none>
I20260807 15:19:30.534348 31511 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:30.534443 31511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:30.534705 31511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:30.536887 31511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "6f5a932128c8411085cf51a7547de383"
format_stamp: "Formatted at 2026-08-07 15:19:30 on dist-test-slave-gkw7"
server_key: "c3c4013b34a3731cacc15afe86e7c02a"
server_key_iv: "84598951608b168a8be547fd4a016ba8"
server_key_version: "encryptionkey@0"
I20260807 15:19:30.537492 31511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "6f5a932128c8411085cf51a7547de383"
format_stamp: "Formatted at 2026-08-07 15:19:30 on dist-test-slave-gkw7"
server_key: "c3c4013b34a3731cacc15afe86e7c02a"
server_key_iv: "84598951608b168a8be547fd4a016ba8"
server_key_version: "encryptionkey@0"
I20260807 15:19:30.541409 31511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 15:19:30.544193 31531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:30.545702 31511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:19:30.545871 31511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "6f5a932128c8411085cf51a7547de383"
format_stamp: "Formatted at 2026-08-07 15:19:30 on dist-test-slave-gkw7"
server_key: "c3c4013b34a3731cacc15afe86e7c02a"
server_key_iv: "84598951608b168a8be547fd4a016ba8"
server_key_version: "encryptionkey@0"
I20260807 15:19:30.545996 31511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:30.561681 31511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:30.564858 31511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:30.565070 31511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:30.573006 31511 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:41005
I20260807 15:19:30.573029 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:41005 every 8 connection(s)
I20260807 15:19:30.574111 31511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:30.577034 31619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:30.583124 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31511
I20260807 15:19:30.583314 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:30.583453 31619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383: Bootstrap starting.
I20260807 15:19:30.583626 32667 external_mini_cluster.cc:1428] Setting key e9ee2b111e89593686eb70d4accdea00
I20260807 15:19:30.586267 31619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:30.587078 31619 log.cc:824] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:30.589118 31619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383: No bootstrap required, opened a new log
I20260807 15:19:30.592983 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:30.585155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57327 (local address 127.31.230.254:41005)
0807 15:19:30.585690 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:30.585707 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:30.585747 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:19:30.586483 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:30.586503 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:30.586875 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:30.587276 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:30.587688 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:30.588669 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:30.589697 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:30.590184 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:30.590379 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:30.591269 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:30.591274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:30.591286 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:30.591683 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:30.591738 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:30.591762 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:30.591877 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:30.592025 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:19:30.592244 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":139,"threads_started":1}
I20260807 15:19:30.593010 31619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5a932128c8411085cf51a7547de383" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41005 } }
I20260807 15:19:30.593339 31619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:30.593429 31619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5a932128c8411085cf51a7547de383, State: Initialized, Role: FOLLOWER
I20260807 15:19:30.593901 31619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5a932128c8411085cf51a7547de383" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41005 } }
I20260807 15:19:30.594058 31619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:30.594144 31619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:30.594273 31619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:30.595357 31619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5a932128c8411085cf51a7547de383" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41005 } }
I20260807 15:19:30.595710 31619 leader_election.cc:302] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f5a932128c8411085cf51a7547de383; no voters:
I20260807 15:19:30.596089 31619 leader_election.cc:288] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:30.596153 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:30.596215 31625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:30.596437 31625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [term 1 LEADER]: Becoming Leader. State: Replica: 6f5a932128c8411085cf51a7547de383, State: Running, Role: LEADER
I20260807 15:19:30.596741 31625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5a932128c8411085cf51a7547de383" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41005 } }
I20260807 15:19:30.597323 31619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:30.598500 31626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f5a932128c8411085cf51a7547de383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5a932128c8411085cf51a7547de383" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41005 } } }
I20260807 15:19:30.598647 31626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:30.598891 31626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f5a932128c8411085cf51a7547de383. Latest consensus state: current_term: 1 leader_uuid: "6f5a932128c8411085cf51a7547de383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5a932128c8411085cf51a7547de383" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 41005 } } }
I20260807 15:19:30.598971 31626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:30.599280 31640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:30.602599 31640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:30.603380 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:30.598896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56173 (local address 127.31.230.254:41005)
0807 15:19:30.599010 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:30.599015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:30.599325 (+ 310us) server_negotiation.cc:407] Connection header received
0807 15:19:30.599518 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:30.599522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:30.599565 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:30.599614 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:30.600128 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:30.600786 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:30.601755 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:30.602004 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:30.602127 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:30.602935 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:30.602939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:30.602942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:30.602954 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:30.602990 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:30.603001 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:30.603083 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:30.603197 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:19:30.603247 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:19:30.608847 31640 catalog_manager.cc:1347] Generated new cluster ID: d34f4b5fd02b4d15ba34cdf346bc4a24
I20260807 15:19:30.608939 31640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:30.645026 31640 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:30.646431 31640 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:30.658245 31640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f5a932128c8411085cf51a7547de383: Generated new TSK 0
I20260807 15:19:30.659216 31640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:30.673600 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31511
2026-08-07T15:19:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:30 dist-test-slave-gkw7 krb5kdc[31659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:30 dist-test-slave-gkw7 krb5kdc[31659](info): set up 2 sockets
Aug 07 15:19:30 dist-test-slave-gkw7 krb5kdc[31659](info): commencing operation
krb5kdc: starting...
W20260807 15:19:31.925364 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:32 dist-test-slave-gkw7 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:32.065150 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38409
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:38409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:32.183252 31681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:32.183614 31681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:32.183696 31681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:32.187932 31681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:32.188035 31681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:32.188170 31681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:32.188212 31681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:32.188249 31681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:32.193770 31681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:38409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:32.194911 31681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:32.196002 31681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:32.204931 31689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:32.204919 31692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:32.204983 31688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:32.205085 31681 server_base.cc:1034] running on GCE node
I20260807 15:19:32.205897 31681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:32.207110 31681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:32.208293 31681 hybrid_clock.cc:648] HybridClock initialized: now 1786115972208273 us; error 32 us; skew 500 ppm
Aug 07 15:19:32 dist-test-slave-gkw7 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115972, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:32.211206 31681 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:32.212229 31681 webserver.cc:460] Webserver started at http://127.31.230.254:40747/ using document root <none> and password file <none>
I20260807 15:19:32.212839 31681 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:32.212903 31681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:32.213174 31681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:32.215423 31681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "86e1ea4625064ce2b718c3802294fcfd"
format_stamp: "Formatted at 2026-08-07 15:19:32 on dist-test-slave-gkw7"
server_key: "251710cece31134c5d73b02f99f2a1b4"
server_key_iv: "dd8880837f7b14574fef8b0a3954b436"
server_key_version: "encryptionkey@0"
I20260807 15:19:32.216020 31681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "86e1ea4625064ce2b718c3802294fcfd"
format_stamp: "Formatted at 2026-08-07 15:19:32 on dist-test-slave-gkw7"
server_key: "251710cece31134c5d73b02f99f2a1b4"
server_key_iv: "dd8880837f7b14574fef8b0a3954b436"
server_key_version: "encryptionkey@0"
I20260807 15:19:32.220100 31681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:19:32.222792 31701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:32.224181 31681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:19:32.224352 31681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "86e1ea4625064ce2b718c3802294fcfd"
format_stamp: "Formatted at 2026-08-07 15:19:32 on dist-test-slave-gkw7"
server_key: "251710cece31134c5d73b02f99f2a1b4"
server_key_iv: "dd8880837f7b14574fef8b0a3954b436"
server_key_version: "encryptionkey@0"
I20260807 15:19:32.224484 31681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:32.229676 31681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:32.232949 31681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:32.233177 31681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:32.240576 31681 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:38409
I20260807 15:19:32.240607 31783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:38409 every 8 connection(s)
I20260807 15:19:32.241636 31681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:32.243602 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31681
I20260807 15:19:32.243778 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:32.244104 32667 external_mini_cluster.cc:1428] Setting key 0f3d3ae4e41b396677599a05b3d88b9e
I20260807 15:19:32.245162 31785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:32.249858 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:32.245767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32817 (local address 127.31.230.254:38409)
0807 15:19:32.246288 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:32.246305 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:32.246342 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:19:32.246991 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:32.247008 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:32.247346 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:32.247717 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:32.248589 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:32.248594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:32.248606 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:32.248617 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:32.248671 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:32.248691 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:32.248855 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:32.248927 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 15:19:32.249094 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":157,"threads_started":1}
I20260807 15:19:32.252648 31785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd: Bootstrap starting.
I20260807 15:19:32.253086 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:32.255200 31785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:32.255954 31785 log.cc:824] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:32.257243 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:32.255264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42377 (local address 127.31.230.254:38409)
0807 15:19:32.255384 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:32.255388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:32.255718 (+ 330us) server_negotiation.cc:407] Connection header received
0807 15:19:32.255874 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:32.255877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:32.255913 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:32.256002 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:32.256789 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:32.256792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:32.256794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:19:32.256804 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:32.256831 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:19:32.256841 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:32.256965 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:32.257084 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:19:32.257126 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:19:32.258548 31785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd: No bootstrap required, opened a new log
I20260807 15:19:32.262266 31785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e1ea4625064ce2b718c3802294fcfd" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38409 } }
I20260807 15:19:32.262516 31785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:19:32.262552 31785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e1ea4625064ce2b718c3802294fcfd, State: Initialized, Role: FOLLOWER
I20260807 15:19:32.262938 31785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e1ea4625064ce2b718c3802294fcfd" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38409 } }
I20260807 15:19:32.263057 31785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:19:32.263099 31785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:19:32.263175 31785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:32.264086 31785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e1ea4625064ce2b718c3802294fcfd" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38409 } }
I20260807 15:19:32.264403 31785 leader_election.cc:302] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e1ea4625064ce2b718c3802294fcfd; no voters:
I20260807 15:19:32.264724 31785 leader_election.cc:288] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:19:32.264896 31801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:32.265256 31801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [term 1 LEADER]: Becoming Leader. State: Replica: 86e1ea4625064ce2b718c3802294fcfd, State: Running, Role: LEADER
I20260807 15:19:32.265668 31801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e1ea4625064ce2b718c3802294fcfd" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38409 } }
I20260807 15:19:32.266121 31785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:32.267690 31804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e1ea4625064ce2b718c3802294fcfd. Latest consensus state: current_term: 1 leader_uuid: "86e1ea4625064ce2b718c3802294fcfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e1ea4625064ce2b718c3802294fcfd" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38409 } } }
I20260807 15:19:32.267904 31804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:32.268204 31803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86e1ea4625064ce2b718c3802294fcfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e1ea4625064ce2b718c3802294fcfd" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 38409 } } }
I20260807 15:19:32.269196 31803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd [sys.catalog]: This master's current role is: LEADER
W20260807 15:19:32.271486 31829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:32.271601 31829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:32.271682 31827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:32.272545 31827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:32.278792 31827 catalog_manager.cc:1347] Generated new cluster ID: 26183bd422e24b50aaf41333e5c37c77
I20260807 15:19:32.278875 31827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:32.302467 31827 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:32.302839 31827 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:32.313882 31827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86e1ea4625064ce2b718c3802294fcfd: Generated new TSK 0
I20260807 15:19:32.314674 31827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:19:32.332183 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31681
2026-08-07T15:19:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1658 ms)
[----------] 18 tests from AuthTokenIssuingTest (29737 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:32 dist-test-slave-gkw7 krb5kdc[31836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:32 dist-test-slave-gkw7 krb5kdc[31836](info): set up 2 sockets
Aug 07 15:19:32 dist-test-slave-gkw7 krb5kdc[31836](info): commencing operation
krb5kdc: starting...
W20260807 15:19:33.572420 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:33 dist-test-slave-gkw7 krb5kdc[31836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:33.713477 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:42187
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:42187,127.31.230.253:39391,127.31.230.252:42589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:33.829218 31855 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:19:33.829638 31855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:33.829698 31855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:33.829774 31855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:33.834020 31855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:33.834081 31855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:33.834101 31855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:33.834117 31855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:33.834141 31855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:33.839690 31855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:42187,127.31.230.253:39391,127.31.230.252:42589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:42187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:33.840814 31855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:33.841938 31855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:33.850790 31863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:33.850785 31862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:33.850785 31866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:33.850888 31855 server_base.cc:1034] running on GCE node
I20260807 15:19:33.851632 31855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:33.852736 31855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:33.853906 31855 hybrid_clock.cc:648] HybridClock initialized: now 1786115973853883 us; error 35 us; skew 500 ppm
Aug 07 15:19:33 dist-test-slave-gkw7 krb5kdc[31836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:33.856848 31855 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:33.857827 31855 webserver.cc:460] Webserver started at http://127.31.230.254:43523/ using document root <none> and password file <none>
I20260807 15:19:33.858371 31855 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:33.858431 31855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:33.858613 31855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:33.860666 31855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "00cb23ec649a44d8adf162f5e082d81a"
format_stamp: "Formatted at 2026-08-07 15:19:33 on dist-test-slave-gkw7"
server_key: "0848a23f44abcdb1d53f57e978c3fa7e"
server_key_iv: "01bdab376539adf97a5960254f90343a"
server_key_version: "encryptionkey@0"
I20260807 15:19:33.861191 31855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "00cb23ec649a44d8adf162f5e082d81a"
format_stamp: "Formatted at 2026-08-07 15:19:33 on dist-test-slave-gkw7"
server_key: "0848a23f44abcdb1d53f57e978c3fa7e"
server_key_iv: "01bdab376539adf97a5960254f90343a"
server_key_version: "encryptionkey@0"
I20260807 15:19:33.864970 31855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:19:33.867573 31877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:33.869040 31855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:19:33.869190 31855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "00cb23ec649a44d8adf162f5e082d81a"
format_stamp: "Formatted at 2026-08-07 15:19:33 on dist-test-slave-gkw7"
server_key: "0848a23f44abcdb1d53f57e978c3fa7e"
server_key_iv: "01bdab376539adf97a5960254f90343a"
server_key_version: "encryptionkey@0"
I20260807 15:19:33.869283 31855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:33.880884 31855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:33.884253 31855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:33.884411 31855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:33.892105 31855 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:42187
I20260807 15:19:33.892407 31954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:42187 every 8 connection(s)
I20260807 15:19:33.893150 31855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:33.896123 31955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:33.900107 31955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:33.901692 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31855
I20260807 15:19:33.901865 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:33.902202 32667 external_mini_cluster.cc:1428] Setting key 226288156e81e79bff157dc352e9d054
Aug 07 15:19:33 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.230.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.253@KRBTEST.COM" created.
I20260807 15:19:33.916705 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:33.902708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.31.230.254:42187)
0807 15:19:33.903172 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:33.903193 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:33.903481 (+ 288us) server_negotiation.cc:407] Connection header received
0807 15:19:33.904558 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:33.904562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:33.904750 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:33.904944 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:33.905890 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:33.906769 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:33.907750 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:33.908122 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:33.908294 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:33.911060 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:33.911065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:33.911075 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:33.911096 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:33.913014 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:33.913707 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:33.913713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:33.913714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:33.913833 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:33.914279 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:33.914289 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:33.914291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:33.914732 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:33.914878 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:33.915260 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 15:19:33.915548 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":306,"thread_start_us":144,"threads_started":1}
I20260807 15:19:33.916705 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:33.902709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:42187 (local address 127.0.0.1:45142)
0807 15:19:33.903328 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:33.903365 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:33.904233 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:33.904949 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:33.904954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:33.905428 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:33.905738 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:33.905754 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:33.906899 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:33.906903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:33.907625 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:33.907633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:33.908281 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:33.908285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:33.908379 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:33.909096 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:33.909115 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:33.910900 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:33.913156 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:33.913168 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:33.913179 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:33.913494 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:33.913962 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:33.913965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:33.913966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:33.914165 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:33.914861 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:33.914868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:33.915004 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 15:19:33.915177 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 15:19:33.915606 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":150,"spinlock_wait_cycles":896,"thread_start_us":163,"threads_started":1}
I20260807 15:19:33.919448 31955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:33.920334 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:33.920060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59580)
0807 15:19:33.920189 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:33.920267 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 15:19:33.920639 31881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
I20260807 15:19:33.921595 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:33.921347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59588)
0807 15:19:33.921452 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:33.921498 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:19:33.921837 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:39391: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab.
Entry for principal kudu/127.31.230.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab.
I20260807 15:19:33.933439 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:39391
--webserver_interface=127.31.230.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab
--principal=kudu/127.31.230.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:42187,127.31.230.253:39391,127.31.230.252:42589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:19:33.971108 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } attempt: 1
I20260807 15:19:33.972090 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:33.971768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59592)
0807 15:19:33.971937 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:33.972006 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 15:19:33.972940 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:33.972664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59600)
0807 15:19:33.972791 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:33.972847 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 15:19:33.973070 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:39391: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
I20260807 15:19:34.041275 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } attempt: 2
I20260807 15:19:34.042364 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.041962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59616)
0807 15:19:34.042183 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.042270 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 15:19:34.043298 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.043026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59620)
0807 15:19:34.043172 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.043224 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 15:19:34.043576 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:39391: Network error: Client connection negotiation failed: client connection to 127.31.230.253:39391: connect: Connection refused (error 111)
W20260807 15:19:34.053508 31966 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:19:34.053896 31966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:34.053959 31966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:34.054054 31966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:34.058852 31966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:34.058938 31966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:34.058961 31966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:34.058982 31966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:34.059072 31966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:34.064885 31966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:42187,127.31.230.253:39391,127.31.230.252:42589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab
--principal=kudu/127.31.230.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.230.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:34.066120 31966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:34.067221 31966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:34.076274 31976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:34.076282 31978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:34.076344 31973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:34.076655 31966 server_base.cc:1034] running on GCE node
I20260807 15:19:34.077224 31966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:34.078397 31966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:34.079557 31966 hybrid_clock.cc:648] HybridClock initialized: now 1786115974079541 us; error 35 us; skew 500 ppm
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:34.085632 31966 init.cc:377] Logged in from keytab as kudu/127.31.230.253@KRBTEST.COM (short username kudu)
I20260807 15:19:34.086556 31966 webserver.cc:460] Webserver started at http://127.31.230.253:34645/ using document root <none> and password file <none>
I20260807 15:19:34.087172 31966 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:34.087239 31966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:34.087466 31966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:34.089663 31966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data/instance:
uuid: "307b900c8b824b549abd672966191436"
format_stamp: "Formatted at 2026-08-07 15:19:34 on dist-test-slave-gkw7"
server_key: "0df0e2815912339c14dd594087b7fa80"
server_key_iv: "7ced790eb20337548a2f0cd7a6a00f8c"
server_key_version: "encryptionkey@0"
I20260807 15:19:34.090250 31966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal/instance:
uuid: "307b900c8b824b549abd672966191436"
format_stamp: "Formatted at 2026-08-07 15:19:34 on dist-test-slave-gkw7"
server_key: "0df0e2815912339c14dd594087b7fa80"
server_key_iv: "7ced790eb20337548a2f0cd7a6a00f8c"
server_key_version: "encryptionkey@0"
I20260807 15:19:34.094416 31966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:19:34.097018 31994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:34.098508 31966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:19:34.098685 31966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal
uuid: "307b900c8b824b549abd672966191436"
format_stamp: "Formatted at 2026-08-07 15:19:34 on dist-test-slave-gkw7"
server_key: "0df0e2815912339c14dd594087b7fa80"
server_key_iv: "7ced790eb20337548a2f0cd7a6a00f8c"
server_key_version: "encryptionkey@0"
I20260807 15:19:34.098804 31966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:34.119581 31966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:34.123368 31966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:34.123618 31966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:34.132112 31966 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.253:39391
I20260807 15:19:34.132440 32066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.253:39391 every 8 connection(s)
I20260807 15:19:34.133278 31966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
I20260807 15:19:34.134791 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } attempt: 3
I20260807 15:19:34.136620 32068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:34.140650 32068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:34.141897 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 31966
I20260807 15:19:34.142053 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-1/wal/instance
I20260807 15:19:34.142432 32667 external_mini_cluster.cc:1428] Setting key 27dac8ab733819b63ef7736aad9dd0aa
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:19:34.148798 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.135395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59630)
0807 15:19:34.135544 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.135562 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.135721 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.138376 (+ 2655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.138380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.138632 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.138785 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.138794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.140222 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.140227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.140830 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.140840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.141815 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.141819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.141940 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.142601 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.142620 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.144239 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.146792 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.146796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.146798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.147015 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.147390 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.147393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.147395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.147489 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.148155 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.148158 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.148274 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.148392 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.148622 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":49}
I20260807 15:19:34.150048 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.136191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.31.230.253:39391)
0807 15:19:34.136748 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.136773 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.136824 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:19:34.137657 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.137679 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.138046 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.138449 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.138959 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.140053 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.140989 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.141645 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.141866 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.144433 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.144439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.144448 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.144476 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.146662 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.147149 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.147154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.147159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.147277 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.147600 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.147603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.147605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.148020 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.148166 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.148561 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.148895 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":157,"threads_started":1}
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:34.152184 31955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:34.153030 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.152780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36096)
0807 15:19:34.152908 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.152966 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:19:34.153966 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.153606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36098)
0807 15:19:34.153829 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.153885 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260807 15:19:34.154284 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.230.252@KRBTEST.COM; defaulting to no policy
I20260807 15:19:34.156164 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.143689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45148 (local address 127.31.230.254:42187)
0807 15:19:34.143862 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.143868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.143886 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:19:34.143927 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.143929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.143978 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.144064 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.145349 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.146697 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.148218 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.148760 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.148881 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.152328 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.152332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.152335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.152359 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.154347 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.154836 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.154840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.154842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.154929 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.155231 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.155235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.155236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.155425 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.155533 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.155888 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.156010 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":74}
I20260807 15:19:34.156208 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.142612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:42187 (local address 127.0.0.1:45148)
0807 15:19:34.143182 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.143246 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.143376 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.144163 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.144173 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.145054 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.145209 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.145216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.146853 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.146860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.148031 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.148047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.149318 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.149321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.149435 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.150095 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.150114 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.152113 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.154446 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.154456 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.154463 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.154720 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.155025 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.155029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.155031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.155120 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.155536 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.155545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.155669 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.155818 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.156011 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":173,"threads_started":1}
I20260807 15:19:34.157410 32068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.252@KRBTEST.COM" created.
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:19:34.166728 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.157968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59642)
0807 15:19:34.158083 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.158098 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.158187 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.158519 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.158525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.158727 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.158883 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.158893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.159875 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.159880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.160523 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.160533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.161003 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.161006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.161095 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.161608 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.161624 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.163196 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.165279 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.165281 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.165283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.165468 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.165800 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.165802 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.165804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.165882 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.166336 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.166338 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.166416 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.166498 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.166607 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 15:19:34.166922 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.158266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.31.230.253:39391)
0807 15:19:34.158352 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.158355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.158363 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:19:34.158384 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.158386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.158414 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.158466 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.159011 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.159746 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.160656 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.160893 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.161002 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.163357 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.163364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.163370 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.163403 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.165162 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.165587 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.165590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.165592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.165698 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.165981 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.165987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.165989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.166233 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.166343 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.166594 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.166728 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:19:34.168308 32068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:34.169081 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.168805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36104)
0807 15:19:34.168946 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.169028 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 15:19:34.169365 31996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
I20260807 15:19:34.170395 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.170165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36106)
0807 15:19:34.170272 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.170317 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 15:19:34.170553 32068 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab.
Entry for principal kudu/127.31.230.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab.
I20260807 15:19:34.178180 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:42589
--webserver_interface=127.31.230.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab
--principal=kudu/127.31.230.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:42187,127.31.230.253:39391,127.31.230.252:42589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:19:34.185472 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 1
I20260807 15:19:34.186504 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.186188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36120)
0807 15:19:34.186357 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.186422 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:19:34.187304 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.187072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36130)
0807 15:19:34.187186 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.187246 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:19:34.187440 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
I20260807 15:19:34.219863 32068 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 1
I20260807 15:19:34.220829 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.220511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36136)
0807 15:19:34.220669 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.220746 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 15:19:34.221678 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.221419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36148)
0807 15:19:34.221559 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.221614 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:19:34.221832 32068 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
I20260807 15:19:34.262624 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 2
I20260807 15:19:34.263540 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.263222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36150)
0807 15:19:34.263382 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.263456 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 15:19:34.264451 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.264190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36152)
0807 15:19:34.264350 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.264394 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 15:19:34.264590 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
I20260807 15:19:34.290019 32068 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 2
I20260807 15:19:34.290985 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.290671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36162)
0807 15:19:34.290839 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.290907 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:19:34.291754 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.291535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36166)
0807 15:19:34.291642 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.291696 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:19:34.291887 32068 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
W20260807 15:19:34.299193 32077 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:19:34.299559 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:34.299621 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:34.299703 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:34.304605 32077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:34.304706 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:34.304746 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:34.304790 32077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:34.304816 32077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:34.310633 32077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:42187,127.31.230.253:39391,127.31.230.252:42589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab
--principal=kudu/127.31.230.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:42589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.230.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:34.311748 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:34.312753 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:34.321699 32085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:34.321722 32089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:34.321900 32077 server_base.cc:1034] running on GCE node
W20260807 15:19:34.322158 32086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:34.322816 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:34.323990 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:34.325188 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786115974325171 us; error 33 us; skew 500 ppm
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:34.328128 32077 init.cc:377] Logged in from keytab as kudu/127.31.230.252@KRBTEST.COM (short username kudu)
I20260807 15:19:34.329072 32077 webserver.cc:460] Webserver started at http://127.31.230.252:42895/ using document root <none> and password file <none>
I20260807 15:19:34.329735 32077 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:34.329829 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:34.330075 32077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:34.332221 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data/instance:
uuid: "9362540824d1451fa09f7c3ca982cf7c"
format_stamp: "Formatted at 2026-08-07 15:19:34 on dist-test-slave-gkw7"
server_key: "410a73559fd9a5e50980e0474c06e4f1"
server_key_iv: "f2f6b5f2807bdcfeb5d2f4daff27f34c"
server_key_version: "encryptionkey@0"
I20260807 15:19:34.332837 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal/instance:
uuid: "9362540824d1451fa09f7c3ca982cf7c"
format_stamp: "Formatted at 2026-08-07 15:19:34 on dist-test-slave-gkw7"
server_key: "410a73559fd9a5e50980e0474c06e4f1"
server_key_iv: "f2f6b5f2807bdcfeb5d2f4daff27f34c"
server_key_version: "encryptionkey@0"
I20260807 15:19:34.336895 32077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260807 15:19:34.339660 32097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:34.340909 32077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:19:34.341068 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal
uuid: "9362540824d1451fa09f7c3ca982cf7c"
format_stamp: "Formatted at 2026-08-07 15:19:34 on dist-test-slave-gkw7"
server_key: "410a73559fd9a5e50980e0474c06e4f1"
server_key_iv: "f2f6b5f2807bdcfeb5d2f4daff27f34c"
server_key_version: "encryptionkey@0"
I20260807 15:19:34.341193 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:34.359879 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:34.363170 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:34.363391 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:34.363750 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 3
I20260807 15:19:34.364658 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.364303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36172)
0807 15:19:34.364449 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.364510 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 15:19:34.365513 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.365306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36178)
0807 15:19:34.365414 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.365460 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:19:34.365712 31955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:42589: Network error: Client connection negotiation failed: client connection to 127.31.230.252:42589: connect: Connection refused (error 111)
I20260807 15:19:34.371239 32077 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.252:42589
I20260807 15:19:34.371580 32172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.252:42589 every 8 connection(s)
I20260807 15:19:34.372525 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
I20260807 15:19:34.375560 32173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:34.376255 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 32077
I20260807 15:19:34.376427 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115680745020-32667-0/minicluster-data/master-2/wal/instance
I20260807 15:19:34.376770 32667 external_mini_cluster.cc:1428] Setting key 6b20597fb5f38fcf23aaca6d662ccedb
I20260807 15:19:34.380285 32173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:34.383105 32068 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 3
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:34.393198 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.383607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36204)
0807 15:19:34.383801 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.383819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.383909 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.384225 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.384228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.384420 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.384519 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.384526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.385388 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.385391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.386077 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.386087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.386568 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.386571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.386664 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.387145 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.387163 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.388577 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.391545 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.391547 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.391549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.391750 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.392101 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.392104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.392106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.392196 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.392630 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.392633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.392751 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.392878 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.393024 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 15:19:34.394007 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.382214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.31.230.254:42187)
0807 15:19:34.382364 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.382369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.382698 (+ 329us) server_negotiation.cc:407] Connection header received
0807 15:19:34.382832 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.382835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.382887 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.382982 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.384045 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.384829 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.386237 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.386632 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.386717 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.389895 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.389900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.389905 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.389940 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.391712 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.392341 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.392347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.392352 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.392461 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.392748 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.392752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.392753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.392994 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.393092 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.393738 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.393871 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:19:34.394460 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.383661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.31.230.252:42589)
0807 15:19:34.383993 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.384003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.384017 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:19:34.384047 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.384050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.384087 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.384167 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.384671 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.385271 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.386213 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.386451 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.386560 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.388730 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.388733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.388736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.388769 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.391422 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.391881 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.391885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.391888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.391984 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.392363 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.392366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.392368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.392539 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.392670 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.393041 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.393356 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260807 15:19:34.394469 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.378316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.31.230.252:42589)
0807 15:19:34.378815 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.378831 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.378875 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:19:34.379556 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.379575 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.379930 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.380296 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.381153 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.382424 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.383679 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.384632 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.384796 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.388052 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.388062 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.388077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.388104 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.390145 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.390707 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.390717 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.390725 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.390851 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.391966 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.391970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.391971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.392526 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.393448 (+ 922us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.393521 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.393687 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":300,"server-negotiator.queue_time_us":350,"spinlock_wait_cycles":3712,"thread_start_us":172,"threads_started":1}
I20260807 15:19:34.394469 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.382136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:42187 (local address 127.0.0.1:45162)
0807 15:19:34.382551 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.382602 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.382733 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.383136 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.383144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.383665 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.383884 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.383892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.384979 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.384984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.386058 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.386071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.386921 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.386925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.387063 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.387592 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.387618 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.389727 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.391817 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.391829 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.391837 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.392202 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.392560 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.392563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.392565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.392659 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.393169 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.393177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.393448 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.393666 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.393834 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":1,"thread_start_us":171,"threads_started":1}
I20260807 15:19:34.395853 32173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:34.399237 32068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: Bootstrap starting.
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:19:34.402053 32068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:34.402894 32068 log.cc:824] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:34.405376 32068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: No bootstrap required, opened a new log
I20260807 15:19:34.405607 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.396525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.31.230.253:39391)
0807 15:19:34.396654 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.396661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.396677 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:19:34.396740 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.396743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.396785 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.396864 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.397330 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.398074 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.399155 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.399452 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.399558 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.401826 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.401830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.401833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.401861 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.403603 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.404130 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.404135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.404137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.404227 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.404538 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.404545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.404547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.404763 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.404845 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.405175 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.405288 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:19:34.406234 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.396432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:39391 (local address 127.0.0.1:59654)
0807 15:19:34.396552 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.396568 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.396653 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.396881 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.396883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.397058 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.397188 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.397197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.398230 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.398236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.399031 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.399040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.399575 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.399578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.399682 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.400168 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.400182 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.401663 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.403751 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.403755 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.403758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.404008 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.404334 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.404337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.404339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.404433 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.404870 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.404873 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.404969 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.405933 (+ 964us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.406087 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 15:19:34.406939 32173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:34.408532 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.397667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.31.230.254:42187)
0807 15:19:34.397801 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.397805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.397818 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:19:34.397852 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.397854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.397893 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.397960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.398820 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.399487 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.400596 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.400893 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.401008 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.403965 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.403971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.403974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.404000 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.406070 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.406743 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.406746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.406748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.406837 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.407530 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.407534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.407536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.407775 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.407995 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.408227 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.408349 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:19:34.409608 32068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.409829 32068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 307b900c8b824b549abd672966191436, State: Initialized, Role: FOLLOWER
I20260807 15:19:34.410194 32068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.410746 32184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.410923 32184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:34.411741 32068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115974, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:19:34.416833 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.407492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36208)
0807 15:19:34.407599 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.407612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.407695 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.408078 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.408082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.408257 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.408345 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.408351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.409446 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.409452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.410213 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.410222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.410799 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.410803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.410871 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.411346 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.411359 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.412926 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.415259 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.415265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.415268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.415510 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.415809 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.415813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.415815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.415890 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.416350 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.416352 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.416428 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.416521 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.416663 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:19:34.417306 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.407628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.31.230.252:42589)
0807 15:19:34.407838 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.407844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.407855 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:19:34.407882 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.407884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.407919 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.407984 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.408468 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.409340 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.410339 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.410680 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.411082 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.413066 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.413069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.413074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.413100 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.415145 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.415620 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.415623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.415638 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.415720 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.415971 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.415973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.415975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.416223 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.416317 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.417047 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.417159 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
W20260807 15:19:34.419526 32200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:34.419628 32200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:34.415918 31966 master_runner.cc:186] Error getting master registration for 127.31.230.254:42187: OK, instance_id { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" instance_seqno: 1786115973889977 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:19:34.419955 31966 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:19:34.420158 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.409992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.31.230.253:39391)
0807 15:19:34.410150 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.410154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.410166 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:19:34.410202 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.410204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.410247 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.410322 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.411398 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.411936 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.412787 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.413083 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.413144 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.416170 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.416173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.416176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.416201 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.417932 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.418444 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.418447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.418449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.418540 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.418849 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.418852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.418854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.419060 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.419247 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.419869 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.419986 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 15:19:34.423514 32173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: Bootstrap starting.
I20260807 15:19:34.425037 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:34.426561 32173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:34.427479 32173 log.cc:824] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:34.426522 31919 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "00cb23ec649a44d8adf162f5e082d81a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45170
I20260807 15:19:34.430043 32173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: No bootstrap required, opened a new log
I20260807 15:19:34.434307 32173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.434597 32173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9362540824d1451fa09f7c3ca982cf7c, State: Initialized, Role: FOLLOWER
I20260807 15:19:34.434964 32173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.435585 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.435801 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:34.436494 32173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:19:34.442780 32222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:34.442869 32222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:34.441071 32077 master_runner.cc:186] Error getting master registration for 127.31.230.254:42187: OK, instance_id { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" instance_seqno: 1786115973889977 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:19:34.444429 32077 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:19:34.529883 31955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } attempt: 4
Aug 07 15:19:34 dist-test-slave-gkw7 krb5kdc[31836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115973, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:19:34.540514 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.530605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.31.230.252:42589)
0807 15:19:34.530748 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.530753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.530918 (+ 165us) server_negotiation.cc:407] Connection header received
0807 15:19:34.531283 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.531287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.531352 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.531450 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.532023 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.532551 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.533714 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.533966 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.534092 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.536766 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.536769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.536772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.536806 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.538718 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.539212 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.539216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.539218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.539309 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.539633 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.539636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.539638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.539846 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.539962 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.540247 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.540371 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:19:34.540514 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.530551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:42589 (local address 127.0.0.1:36212)
0807 15:19:34.530802 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:34.530820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:34.530983 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:34.531484 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:34.531487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:34.531749 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:34.531872 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.531878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.532677 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.532682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.533603 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:34.533617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.534102 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.534106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:34.534197 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.534817 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:34.534832 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:34.536590 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:34.538840 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.538844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.538846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.539117 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.539434 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:34.539438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:34.539440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:34.539533 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:34.539951 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:34.539954 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:34.540068 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:19:34.540168 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 15:19:34.540328 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260807 15:19:34.544827 31955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a: Bootstrap starting.
I20260807 15:19:34.548099 31955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:34.549073 31955 log.cc:824] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:34.551501 31955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a: No bootstrap required, opened a new log
I20260807 15:19:34.555097 31955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.555342 31955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cb23ec649a44d8adf162f5e082d81a, State: Initialized, Role: FOLLOWER
I20260807 15:19:34.555706 31955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.556316 32225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.556697 32225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:34.557379 31955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:34.557425 31919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:19:34.557602 31919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:34.559194 31919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.561233 32030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00cb23ec649a44d8adf162f5e082d81a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "307b900c8b824b549abd672966191436"
I20260807 15:19:34.561372 32136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00cb23ec649a44d8adf162f5e082d81a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9362540824d1451fa09f7c3ca982cf7c"
I20260807 15:19:34.561594 32030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:34.562564 32136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:34.563143 32030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cb23ec649a44d8adf162f5e082d81a in term 1.
I20260807 15:19:34.563993 32136 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cb23ec649a44d8adf162f5e082d81a in term 1.
I20260807 15:19:34.564518 31881 leader_election.cc:302] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cb23ec649a44d8adf162f5e082d81a, 307b900c8b824b549abd672966191436; no voters:
W20260807 15:19:34.566876 32239 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:34.566959 32239 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:34.567246 31919 leader_election.cc:288] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [CANDIDATE]: Term 1 election: Requested vote from peers 307b900c8b824b549abd672966191436 (127.31.230.253:39391), 9362540824d1451fa09f7c3ca982cf7c (127.31.230.252:42589)
I20260807 15:19:34.569820 32225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:34.570747 32225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [term 1 LEADER]: Becoming Leader. State: Replica: 00cb23ec649a44d8adf162f5e082d81a, State: Running, Role: LEADER
I20260807 15:19:34.570923 31855 master_runner.cc:263] No leader master found from master 00cb23ec649a44d8adf162f5e082d81a
I20260807 15:19:34.570989 31855 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:19:34.571249 32225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } }
I20260807 15:19:34.573076 32225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cb23ec649a44d8adf162f5e082d81a. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.573199 32225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:34.577791 32249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:34.578979 32249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:19:34.586421 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.573863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.31.230.254:42187)
0807 15:19:34.574000 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.574008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.575667 (+ 1659us) server_negotiation.cc:407] Connection header received
0807 15:19:34.575903 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.575909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.575963 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.576025 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.577209 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.577989 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.579189 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.579539 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.579628 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.582182 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.582186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.582189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.582220 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.584133 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.584741 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.584745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.584747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.584848 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.585214 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.585218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.585220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.585465 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.585642 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.586072 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.586218 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:19:34.597851 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.575221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.31.230.252:42589)
0807 15:19:34.575337 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.575340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.581861 (+ 6521us) server_negotiation.cc:407] Connection header received
0807 15:19:34.582104 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.582108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.582165 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.582225 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.583163 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.583898 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.585124 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.585459 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.585573 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.593680 (+ 8107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.593684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.593687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.593719 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.595562 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.596134 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.596138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.596140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.596238 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.596571 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.596703 (+ 132us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.596712 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.596975 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.597095 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.597514 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.597688 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:19:34.600577 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:34.574718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.31.230.253:39391)
0807 15:19:34.574867 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.574870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.585864 (+ 10994us) server_negotiation.cc:407] Connection header received
0807 15:19:34.586098 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.586101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.586162 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.586228 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.590088 (+ 3860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.590892 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.592081 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:34.592432 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:34.592515 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:34.594655 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:34.594660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:34.594663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:34.594694 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:34.596747 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.597313 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.597318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.597320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.597419 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:34.597801 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:34.597805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:34.597807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:34.598325 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:34.598454 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:34.600244 (+ 1790us) server_negotiation.cc:299] Negotiation successful
0807 15:19:34.600380 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:19:34.605584 32136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [term 1 FOLLOWER]: Refusing update from remote peer 00cb23ec649a44d8adf162f5e082d81a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:34.605786 32030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [term 1 FOLLOWER]: Refusing update from remote peer 00cb23ec649a44d8adf162f5e082d81a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:34.607479 32246 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:34.607995 32246 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [LEADER]: Connected to new peer: Peer: permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:19:34.615109 32104 tablet.cc:2369] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:19:34.618937 32184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cb23ec649a44d8adf162f5e082d81a. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.619127 32184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:34.621029 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cb23ec649a44d8adf162f5e082d81a. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.621196 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:19:34.625319 32001 tablet.cc:2369] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:19:34.634480 32225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.634671 32225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:34.636412 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.636564 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:34.642474 32246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.642640 32246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:34.644081 32184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00cb23ec649a44d8adf162f5e082d81a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cb23ec649a44d8adf162f5e082d81a" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 42187 } } peers { permanent_uuid: "307b900c8b824b549abd672966191436" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 39391 } } peers { permanent_uuid: "9362540824d1451fa09f7c3ca982cf7c" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 42589 } } }
I20260807 15:19:34.644212 32184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:34.646387 32255 mvcc.cc:205] Tried to move back new op lower bound from 7315931031976312832 to 7315931031847321600. Current Snapshot: MvccSnapshot[applied={T|T < 7315931031976312832}]
I20260807 15:19:34.646562 32258 mvcc.cc:205] Tried to move back new op lower bound from 7315931031976312832 to 7315931031847321600. Current Snapshot: MvccSnapshot[applied={T|T < 7315931031976312832 or (T in {7315931031976312832})}]
I20260807 15:19:34.648356 32249 catalog_manager.cc:1347] Generated new cluster ID: db119e87cc55439684d4607c4dca37a6
I20260807 15:19:34.648481 32249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:34.668829 32249 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:34.670033 32249 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:34.678709 32249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00cb23ec649a44d8adf162f5e082d81a: Generated new TSK 0
I20260807 15:19:34.680289 32249 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:19:34.713307 31958 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.711623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.31.230.254:42187)
0807 15:19:34.711768 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.711773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.712041 (+ 268us) server_negotiation.cc:407] Connection header received
0807 15:19:34.712287 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.712293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.712358 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.712418 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.713153 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45192: BlockingRecv error: recv got EOF from 127.0.0.1:45192 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 15:19:34.713711 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.712102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.31.230.253:39391)
0807 15:19:34.712233 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.712238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.712642 (+ 404us) server_negotiation.cc:407] Connection header received
0807 15:19:34.712829 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.712834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.712888 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.712968 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.713588 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59676: BlockingRecv error: recv got EOF from 127.0.0.1:59676 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 15:19:34.713773 32177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.711728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.31.230.252:42589)
0807 15:19:34.711857 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.711861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.712574 (+ 713us) server_negotiation.cc:407] Connection header received
0807 15:19:34.712827 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.712831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.712889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.712945 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.713627 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36226: BlockingRecv error: recv got EOF from 127.0.0.1:36226 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 15:19:34.717206 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.716068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.31.230.253:39391)
0807 15:19:34.716187 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.716192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.716324 (+ 132us) server_negotiation.cc:407] Connection header received
0807 15:19:34.716458 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.716460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.716515 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.716582 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.717032 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59688: BlockingRecv error: recv got EOF from 127.0.0.1:59688 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 15:19:34.721383 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.720555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.31.230.253:39391)
0807 15:19:34.720673 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.720676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.720687 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:19:34.720726 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.720728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.720764 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.720856 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.721260 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59702: BlockingRecv error: recv got EOF from 127.0.0.1:59702 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:19:34.726418 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.725439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.31.230.253:39391)
0807 15:19:34.725602 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.725607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.725621 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:19:34.725768 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.725773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.725833 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.725927 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.726318 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59706: BlockingRecv error: recv got EOF from 127.0.0.1:59706 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 15:19:34.733357 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.732204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.31.230.253:39391)
0807 15:19:34.732366 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.732373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.732485 (+ 112us) server_negotiation.cc:407] Connection header received
0807 15:19:34.732656 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.732659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.732708 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.732809 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.733247 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59720: BlockingRecv error: recv got EOF from 127.0.0.1:59720 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260807 15:19:34.744894 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.743687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.31.230.253:39391)
0807 15:19:34.743826 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.743831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.743964 (+ 133us) server_negotiation.cc:407] Connection header received
0807 15:19:34.744132 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.744136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.744193 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.744285 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.744745 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59722: BlockingRecv error: recv got EOF from 127.0.0.1:59722 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 15:19:34.764823 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.763693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.31.230.253:39391)
0807 15:19:34.763829 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.763834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.763988 (+ 154us) server_negotiation.cc:407] Connection header received
0807 15:19:34.764111 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.764116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.764172 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.764275 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.764699 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59734: BlockingRecv error: recv got EOF from 127.0.0.1:59734 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 15:19:34.800335 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.799156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.31.230.253:39391)
0807 15:19:34.799354 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.799360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.799497 (+ 137us) server_negotiation.cc:407] Connection header received
0807 15:19:34.799629 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.799632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.799685 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.799761 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.800201 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59750: BlockingRecv error: recv got EOF from 127.0.0.1:59750 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 15:19:34.868207 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.866863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.31.230.253:39391)
0807 15:19:34.867037 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.867042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.867249 (+ 207us) server_negotiation.cc:407] Connection header received
0807 15:19:34.867392 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.867395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.867447 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.867525 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.868032 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59764: BlockingRecv error: recv got EOF from 127.0.0.1:59764 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 15:19:34.999790 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:34.998541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.31.230.253:39391)
0807 15:19:34.998712 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:34.998720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:34.998842 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:19:34.998970 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:34.998972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:34.999025 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:34.999102 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:34.999611 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59774: BlockingRecv error: recv got EOF from 127.0.0.1:59774 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 15:19:35.259763 32070 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:35.258216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.31.230.253:39391)
0807 15:19:35.258403 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:35.258409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:35.258569 (+ 160us) server_negotiation.cc:407] Connection header received
0807 15:19:35.258871 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:35.258875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:35.258986 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:35.259067 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:35.259590 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59784: BlockingRecv error: recv got EOF from 127.0.0.1:59784 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 15:19:35.421520 32200 catalog_manager.cc:1259] Loaded cluster ID: db119e87cc55439684d4607c4dca37a6
I20260807 15:19:35.421674 32200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: loading cluster ID for follower catalog manager: success
I20260807 15:19:35.424530 32200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: acquiring CA information for follower catalog manager: success
I20260807 15:19:35.426198 32200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 307b900c8b824b549abd672966191436: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:19:35.444337 32222 catalog_manager.cc:1259] Loaded cluster ID: db119e87cc55439684d4607c4dca37a6
I20260807 15:19:35.444432 32222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: loading cluster ID for follower catalog manager: success
I20260807 15:19:35.446442 32222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: acquiring CA information for follower catalog manager: success
I20260807 15:19:35.447718 32222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9362540824d1451fa09f7c3ca982cf7c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:19:35.778985 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:35.774624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59796 (local address 127.31.230.253:39391)
0807 15:19:35.774945 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:35.774952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:35.774972 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:19:35.775222 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:35.775225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:35.775361 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:35.775491 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:35.775794 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.776342 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.777290 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.777581 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.777705 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:35.778055 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:35.778524 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:35.778686 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:35.778733 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 15:19:35.778798 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
I20260807 15:19:35.786936 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:35.782901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36240 (local address 127.31.230.252:42589)
0807 15:19:35.783139 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:35.783143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:35.783164 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:19:35.783316 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:35.783319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:35.783443 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:35.783546 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:35.783829 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.784374 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.785345 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.785633 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.785716 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:35.786033 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:35.786472 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:35.786566 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:35.786670 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 15:19:35.786723 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":94,"threads_started":1}
I20260807 15:19:35.795032 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:35.790616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.31.230.253:39391)
0807 15:19:35.790781 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:35.790785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:35.791259 (+ 474us) server_negotiation.cc:407] Connection header received
0807 15:19:35.791457 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:35.791460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:35.791518 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:35.791619 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:35.791894 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.792652 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.793601 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.793896 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.794018 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:35.794575 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:35.794661 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:35.794730 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:35.794822 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 15:19:35.794874 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:19:35.795342 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:35.790406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.31.230.252:42589)
0807 15:19:35.790527 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:35.790533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:35.790789 (+ 256us) server_negotiation.cc:407] Connection header received
0807 15:19:35.791002 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:35.791005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:35.791055 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:35.791139 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:35.791745 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.792523 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.793716 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.794056 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.794187 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:35.794480 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:35.794564 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:35.794648 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:35.795036 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 15:19:35.795212 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:19:35.795610 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:35.790239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.31.230.254:42187)
0807 15:19:35.790548 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:35.790554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:35.790660 (+ 106us) server_negotiation.cc:407] Connection header received
0807 15:19:35.790880 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:35.790883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:35.791089 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:35.791223 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:35.791667 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.792284 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.793500 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:35.793868 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:35.793972 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:35.794384 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:35.794792 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:35.794874 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:35.795304 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 15:19:35.795370 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":98,"threads_started":1}
I20260807 15:19:35.799012 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31855
I20260807 15:19:35.814767 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 31966
I20260807 15:19:35.830111 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 32077
2026-08-07T15:19:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3498 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:19:35 dist-test-slave-gkw7 krb5kdc[32383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:19:35 dist-test-slave-gkw7 krb5kdc[32383](info): set up 2 sockets
Aug 07 15:19:35 dist-test-slave-gkw7 krb5kdc[32383](info): commencing operation
krb5kdc: starting...
W20260807 15:19:37.062947 32667 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/test-user.keytab.
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:19:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:19:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.230.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
Entry for principal kudu/127.31.230.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab.
I20260807 15:19:37.205173 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:35721
--webserver_interface=127.31.230.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:35721,127.31.230.253:33263,127.31.230.252:45685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:19:37.331843 32401 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:19:37.332204 32401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:37.332283 32401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:37.332363 32401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:37.336397 32401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:37.336519 32401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:37.336565 32401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:37.336625 32401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:37.336663 32401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:37.342391 32401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:35721,127.31.230.253:33263,127.31.230.252:45685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.254.keytab
--principal=kudu/127.31.230.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.254:35721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.230.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:37.343674 32401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:37.344781 32401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:37.353645 32408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:37.353819 32401 server_base.cc:1034] running on GCE node
W20260807 15:19:37.353639 32409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:37.353639 32411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:37.354574 32401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:37.355722 32401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:37.356930 32401 hybrid_clock.cc:648] HybridClock initialized: now 1786115977356908 us; error 37 us; skew 500 ppm
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:37.359784 32401 init.cc:377] Logged in from keytab as kudu/127.31.230.254@KRBTEST.COM (short username kudu)
I20260807 15:19:37.360749 32401 webserver.cc:460] Webserver started at http://127.31.230.254:41723/ using document root <none> and password file <none>
I20260807 15:19:37.361332 32401 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:37.361428 32401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:37.361718 32401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:37.363832 32401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data/instance:
uuid: "1b697c40e41644f888019305a64e3626"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "047919123accc994af233e8de13d33c9"
server_key_iv: "334a72720add2d58bac0fd112db94121"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.364436 32401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal/instance:
uuid: "1b697c40e41644f888019305a64e3626"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "047919123accc994af233e8de13d33c9"
server_key_iv: "334a72720add2d58bac0fd112db94121"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.368275 32401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:19:37.370896 32422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:37.372263 32401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:37.372412 32401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal
uuid: "1b697c40e41644f888019305a64e3626"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "047919123accc994af233e8de13d33c9"
server_key_iv: "334a72720add2d58bac0fd112db94121"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.372534 32401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:37.388914 32401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:37.392195 32401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:37.392413 32401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:37.400331 32401 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.254:35721
I20260807 15:19:37.400635 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.254:35721 every 8 connection(s)
I20260807 15:19:37.401444 32401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/data/info.pb
I20260807 15:19:37.403515 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 32401
I20260807 15:19:37.403671 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-0/wal/instance
I20260807 15:19:37.403960 32667 external_mini_cluster.cc:1428] Setting key 2e53333810e6e3be850914a7cb1719e3
I20260807 15:19:37.404631 32504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:37.408838 32504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.230.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.253@KRBTEST.COM" created.
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:37.427399 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.411435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:35721 (local address 127.0.0.1:34994)
0807 15:19:37.412103 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.412169 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.413124 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.414051 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.414056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.414453 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.414730 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.414744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.416194 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.416198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.417694 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.417711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.418782 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.418786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.418875 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.419588 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.419609 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.421353 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.423996 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.424006 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.424015 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.424279 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.424691 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.424693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.424695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.424823 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.425733 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.425739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.425883 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.426049 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.426353 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"spinlock_wait_cycles":6272,"thread_start_us":176,"threads_started":1}
I20260807 15:19:37.427452 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.411435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.31.230.254:35721)
0807 15:19:37.411962 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.411986 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.412301 (+ 315us) server_negotiation.cc:407] Connection header received
0807 15:19:37.413633 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.413639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.413838 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.414000 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.414943 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.416035 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.417846 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.418596 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.418798 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.421569 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.421580 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.421598 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.421634 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.423865 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.424411 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.424421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.424425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.424583 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.424933 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.424940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.424944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.425601 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.425754 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.426190 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.426464 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":346,"thread_start_us":152,"threads_started":1}
I20260807 15:19:37.430284 32504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:37.431200 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.430784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54552)
0807 15:19:37.431032 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.431133 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260807 15:19:37.431427 32424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
I20260807 15:19:37.432512 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.432291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54556)
0807 15:19:37.432406 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.432456 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:19:37.432698 32504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:33263: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab.
Entry for principal kudu/127.31.230.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab.
I20260807 15:19:37.437652 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:33263
--webserver_interface=127.31.230.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab
--principal=kudu/127.31.230.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:35721,127.31.230.253:33263,127.31.230.252:45685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:19:37.481974 32504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } attempt: 1
I20260807 15:19:37.482993 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.482582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54566)
0807 15:19:37.482792 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.482887 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 15:19:37.483970 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.483636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54574)
0807 15:19:37.483791 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.483872 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 15:19:37.484154 32504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:33263: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
I20260807 15:19:37.552356 32504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } attempt: 2
I20260807 15:19:37.553308 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.552939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54588)
0807 15:19:37.553130 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.553210 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 15:19:37.554162 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.553921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54602)
0807 15:19:37.554042 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.554099 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:19:37.554348 32504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.253:33263: Network error: Client connection negotiation failed: client connection to 127.31.230.253:33263: connect: Connection refused (error 111)
W20260807 15:19:37.559898 32516 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:19:37.560179 32516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:37.560256 32516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:37.560326 32516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:37.564559 32516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:37.564617 32516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:37.564636 32516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:37.564685 32516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:37.564719 32516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:37.570176 32516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:35721,127.31.230.253:33263,127.31.230.252:45685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.253.keytab
--principal=kudu/127.31.230.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.253:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.230.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:37.571280 32516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:37.572355 32516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:37.581831 32529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:37.581831 32524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:37.581915 32523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:37.582115 32516 server_base.cc:1034] running on GCE node
I20260807 15:19:37.582692 32516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:37.583918 32516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:37.585155 32516 hybrid_clock.cc:648] HybridClock initialized: now 1786115977585138 us; error 32 us; skew 500 ppm
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:37.588171 32516 init.cc:377] Logged in from keytab as kudu/127.31.230.253@KRBTEST.COM (short username kudu)
I20260807 15:19:37.589134 32516 webserver.cc:460] Webserver started at http://127.31.230.253:34261/ using document root <none> and password file <none>
I20260807 15:19:37.589741 32516 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:37.589807 32516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:37.589983 32516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:37.592036 32516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data/instance:
uuid: "e0c6dc9ff96843489c954e21709ebb1f"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "0c76593482d8fee2e513021979b8f868"
server_key_iv: "e646b8b068e16eeffff535f4301fb237"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.592576 32516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal/instance:
uuid: "e0c6dc9ff96843489c954e21709ebb1f"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "0c76593482d8fee2e513021979b8f868"
server_key_iv: "e646b8b068e16eeffff535f4301fb237"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.596422 32516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 15:19:37.598954 32541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:37.600338 32516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:37.600567 32516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal
uuid: "e0c6dc9ff96843489c954e21709ebb1f"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "0c76593482d8fee2e513021979b8f868"
server_key_iv: "e646b8b068e16eeffff535f4301fb237"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.600696 32516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:37.610613 32516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:37.613911 32516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:37.614130 32516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:37.621791 32516 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.253:33263
I20260807 15:19:37.622133 32624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.253:33263 every 8 connection(s)
I20260807 15:19:37.622848 32516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/data/info.pb
I20260807 15:19:37.625667 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 32516
I20260807 15:19:37.625809 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-1/wal/instance
I20260807 15:19:37.626039 32625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:37.626145 32667 external_mini_cluster.cc:1428] Setting key 265c731ea8f2d4c8cf3928335392d242
I20260807 15:19:37.630142 32625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.230.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.230.252@KRBTEST.COM" created.
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:37.645540 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.632177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35000 (local address 127.31.230.254:35721)
0807 15:19:37.632381 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.632388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.633232 (+ 844us) server_negotiation.cc:407] Connection header received
0807 15:19:37.634306 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.634313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.634390 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.634512 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.635825 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.636454 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.637819 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.638196 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.638289 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.641128 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.641131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.641134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.641160 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.643055 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.643648 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.643651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.643653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.643772 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.644220 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.644223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.644224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.644414 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.644496 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.645256 (+ 760us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.645371 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:19:37.645576 32504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } attempt: 3
I20260807 15:19:37.646873 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.632547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:35721 (local address 127.0.0.1:35000)
0807 15:19:37.633045 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.633102 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.634067 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.634690 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.634725 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.635161 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.635668 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.635683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.636605 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.636608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.637663 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.637674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.638376 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.638383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.638582 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.639262 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.639280 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.640934 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.643207 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.643212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.643226 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.643488 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.643893 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.643899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.643902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.644103 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.644535 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.644545 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.644926 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.645195 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.645635 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":154,"threads_started":1}
I20260807 15:19:37.648591 32625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.230.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab.
Entry for principal kudu/127.31.230.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab.
I20260807 15:19:37.657277 32667 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
/tmp/dist-test-taskLoCTfz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:45685
--webserver_interface=127.31.230.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab
--principal=kudu/127.31.230.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.230.212:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.230.254:35721,127.31.230.253:33263,127.31.230.252:45685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:19:37.657721 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.646189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54606)
0807 15:19:37.646381 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.646396 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.646480 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.647514 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.647518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.647717 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.648253 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.648261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.649638 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.649643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.650247 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.650257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.651027 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.651033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.651169 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.651695 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.651710 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.653149 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.655525 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.655528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.655529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.655789 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.656158 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.656161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.656163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.656250 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.656901 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.656904 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.657010 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.657337 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.657501 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 15:19:37.657841 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.646311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.31.230.253:33263)
0807 15:19:37.646750 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.646760 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.646779 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:19:37.646814 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.646818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.647034 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.647526 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.648386 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.649441 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.650379 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.650899 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.651013 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.653278 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.653293 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.653305 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.653326 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.655426 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.655912 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.655919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.655921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.656020 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.656393 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.656397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.656399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.656773 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.656940 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.657461 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.657654 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":145,"threads_started":1}
I20260807 15:19:37.660526 32504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:37.661468 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.661146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47408)
0807 15:19:37.661297 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.661367 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:19:37.662333 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.662074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47420)
0807 15:19:37.662221 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.662270 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 15:19:37.662560 32504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:45685: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
I20260807 15:19:37.663569 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.649180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.31.230.253:33263)
0807 15:19:37.649461 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.649468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.649479 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:19:37.649533 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.649539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.649617 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.649732 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.650098 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.650899 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.652227 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.652538 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.652635 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.655007 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.655011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.655013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.655038 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.661458 (+ 6420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.662110 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.662117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.662121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.662222 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.662523 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.662527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.662529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.662780 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.662867 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.663202 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.663429 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":222,"thread_start_us":139,"threads_started":1}
I20260807 15:19:37.663589 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.649087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54610)
0807 15:19:37.649267 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.649282 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.649352 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.649741 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.649744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.649913 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.649987 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.649992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.651424 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.651428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.652071 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.652083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.652673 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.652677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.652799 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.653332 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.653346 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.654849 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.661687 (+ 6838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.661691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.661692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.661985 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.662336 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.662342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.662347 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.662434 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.662889 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.662895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.663012 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.663143 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.663329 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 15:19:37.664481 32625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:37.665333 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.664962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47428)
0807 15:19:37.665138 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.665214 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 15:19:37.665647 32544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
I20260807 15:19:37.666635 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.666416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47436)
0807 15:19:37.666532 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.666576 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 15:19:37.666960 32625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:45685: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
I20260807 15:19:37.693737 32504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } attempt: 1
I20260807 15:19:37.694674 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.694346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47448)
0807 15:19:37.694511 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.694575 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 15:19:37.695466 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.695235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47464)
0807 15:19:37.695352 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.695406 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:19:37.695619 32504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:45685: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
I20260807 15:19:37.716210 32625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } attempt: 1
I20260807 15:19:37.717123 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.716811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47476)
0807 15:19:37.716973 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.717039 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 15:19:37.717934 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.717707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47492)
0807 15:19:37.717819 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.717878 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 15:19:37.718076 32625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:45685: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
I20260807 15:19:37.770793 32504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } attempt: 2
I20260807 15:19:37.771821 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.771385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47504)
0807 15:19:37.771615 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.771707 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 15:19:37.772652 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.772399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47506)
0807 15:19:37.772529 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.772597 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 15:19:37.772794 32504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:45685: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
W20260807 15:19:37.776620 32635 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:19:37.776949 32635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:19:37.777011 32635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:19:37.777099 32635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:19:37.781263 32635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:19:37.781364 32635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:19:37.781414 32635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:19:37.781446 32635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:19:37.781478 32635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:19:37.786254 32625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } attempt: 2
I20260807 15:19:37.787081 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.786783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47522)
0807 15:19:37.786930 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.786999 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:19:37.787045 32635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.230.212:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.230.254:35721,127.31.230.253:33263,127.31.230.252:45685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/krb5kdc/kudu_127.31.230.252.keytab
--principal=kudu/127.31.230.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.230.252:45685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.230.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:19:37.787891 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.787664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47528)
0807 15:19:37.787781 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.787833 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 15:19:37.788019 32625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.230.252:45685: Network error: Client connection negotiation failed: client connection to 127.31.230.252:45685: connect: Connection refused (error 111)
I20260807 15:19:37.788347 32635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:19:37.789393 32635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:19:37.798672 32643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:37.798707 32641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:19:37.798656 32645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:19:37.798830 32635 server_base.cc:1034] running on GCE node
I20260807 15:19:37.799397 32635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:19:37.800514 32635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:19:37.801681 32635 hybrid_clock.cc:648] HybridClock initialized: now 1786115977801652 us; error 44 us; skew 500 ppm
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:19:37.804800 32635 init.cc:377] Logged in from keytab as kudu/127.31.230.252@KRBTEST.COM (short username kudu)
I20260807 15:19:37.805768 32635 webserver.cc:460] Webserver started at http://127.31.230.252:36143/ using document root <none> and password file <none>
I20260807 15:19:37.806293 32635 fs_manager.cc:359] Metadata directory not provided
I20260807 15:19:37.806354 32635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:19:37.806516 32635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:19:37.808475 32635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data/instance:
uuid: "205cd3b362744eba9eea3c6133498428"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "2eef802e7e3f65aefd5417b534506e97"
server_key_iv: "f2bd396fdd419cac7df26eb32394eb3b"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.808990 32635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal/instance:
uuid: "205cd3b362744eba9eea3c6133498428"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "2eef802e7e3f65aefd5417b534506e97"
server_key_iv: "f2bd396fdd419cac7df26eb32394eb3b"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.812597 32635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 15:19:37.815026 32656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:19:37.816250 32635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:19:37.816381 32635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal
uuid: "205cd3b362744eba9eea3c6133498428"
format_stamp: "Formatted at 2026-08-07 15:19:37 on dist-test-slave-gkw7"
server_key: "2eef802e7e3f65aefd5417b534506e97"
server_key_iv: "f2bd396fdd419cac7df26eb32394eb3b"
server_key_version: "encryptionkey@0"
I20260807 15:19:37.816483 32635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:19:37.834215 32635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:19:37.838078 32635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:19:37.838302 32635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:19:37.845160 32635 rpc_server.cc:307] RPC server started. Bound to: 127.31.230.252:45685
I20260807 15:19:37.845475 32745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.230.252:45685 every 8 connection(s)
I20260807 15:19:37.846271 32635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/data/info.pb
I20260807 15:19:37.849080 32747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:19:37.852788 32747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:37.856339 32667 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu as pid 32635
I20260807 15:19:37.856494 32667 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoCTfz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115680745020-32667-0/minicluster-data/master-2/wal/instance
I20260807 15:19:37.856863 32667 external_mini_cluster.cc:1428] Setting key 04c5aa0454154f84d77e3d9f1e7a44bd
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:19:37.867398 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.854679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.31.230.254:35721)
0807 15:19:37.854868 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.854874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.855756 (+ 882us) server_negotiation.cc:407] Connection header received
0807 15:19:37.856777 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.856780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.856839 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.856945 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.858476 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.859128 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.860335 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.860555 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.860647 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.863434 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.863437 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.863439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.863466 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.865178 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.865789 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.865793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.865800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.865880 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.866245 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.866248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.866250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.866456 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.866572 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.867085 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.867214 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 15:19:37.868552 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.855077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.254:35721 (local address 127.0.0.1:35014)
0807 15:19:37.855584 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.855637 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.856505 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.857122 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.857130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.857647 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.858294 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.858310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.859255 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.859258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.860196 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.860209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.860728 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.860731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.860933 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.861642 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.861660 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.863277 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.865295 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.865306 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.865317 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.865691 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.865979 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.865983 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.865985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.866137 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.866570 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.866577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.866831 (+ 254us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.867022 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.867347 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":54,"thread_start_us":129,"threads_started":1}
I20260807 15:19:37.869839 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.857636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.31.230.252:45685)
0807 15:19:37.857973 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.857982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.858004 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:19:37.858130 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.858134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.858281 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.858440 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.859193 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.860099 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.861205 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.861737 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.861862 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.864996 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.865003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.865014 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.865038 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.866989 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.867762 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.867767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.867768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.867853 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.868183 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.868188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.868190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.868431 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.869364 (+ 933us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.869481 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.869685 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":234,"thread_start_us":163,"threads_started":1}
I20260807 15:19:37.870165 32747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:37.871965 32504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } attempt: 3
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.254@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:19:37.879204 32625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } attempt: 3
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.254@KRBTEST.COM
I20260807 15:19:37.880409 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.870776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.253:33263 (local address 127.0.0.1:54618)
0807 15:19:37.870924 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.870941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.871025 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.871310 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.871313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.871506 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.871631 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.871638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.872807 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.872812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.873698 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.873711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.874317 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.874320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.874432 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.874899 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.874912 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.876469 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.878727 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.878730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.878731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.878916 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.879296 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.879303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.879305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.879404 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.879882 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.879886 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.879990 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.880124 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.880252 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 15:19:37.880463 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.870913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54618 (local address 127.31.230.253:33263)
0807 15:19:37.871052 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.871059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.871081 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:19:37.871148 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.871151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.871194 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.871294 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.871816 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.872684 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.873821 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.874193 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.874308 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.876685 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.876692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.876697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.876727 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.878596 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.879025 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.879029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.879031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.879130 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.879499 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.879506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.879510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.879736 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.879833 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.880207 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.880328 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:19:37.882207 32747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:19:37.884914 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.873019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47550)
0807 15:19:37.873186 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.873197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.873257 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.873887 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.873890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.874108 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.874208 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.874215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.875334 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.875340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.876496 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.876508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.877255 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.877261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.877351 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.877867 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.877885 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.880204 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.882767 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.882770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.882773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.883446 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.883786 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.883790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.883792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.883890 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.884359 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.884362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.884460 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.884559 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.884706 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":19}
I20260807 15:19:37.885887 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.872110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.31.230.254:35721)
0807 15:19:37.872244 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.872248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.872337 (+ 89us) server_negotiation.cc:407] Connection header received
0807 15:19:37.872417 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.872420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.872468 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.872545 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.873342 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.873973 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.875198 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.875654 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.875758 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.880870 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.880873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.880876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.880899 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.883506 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.884024 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.884027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.884029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.884113 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.884424 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.884427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.884429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.884639 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.885508 (+ 869us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.885615 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.885735 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":55}
I20260807 15:19:37.886354 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.873060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.31.230.252:45685)
0807 15:19:37.873675 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.873680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.873694 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:19:37.873727 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.873732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.873784 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.873866 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.874351 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.875173 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.876648 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.877103 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.877586 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.880575 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.880580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.880582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.880605 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.882643 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.883567 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.883570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.883572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.883673 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.884005 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.884008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.884010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.884215 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.884311 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.885447 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.885637 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":536}
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.253@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.230.252@KRBTEST.COM for kudu/127.31.230.252@KRBTEST.COM
I20260807 15:19:37.890416 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626: Bootstrap starting.
I20260807 15:19:37.892558 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.882733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47568)
0807 15:19:37.882848 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.882862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.882935 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.883546 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.883552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.883715 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.883806 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.883813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.884814 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.884817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.885635 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.885646 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.886627 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.886630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.886711 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.887597 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.887609 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.888932 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.890995 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.890999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.891001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.891256 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.891565 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.891568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.891569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.891649 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.892139 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.892141 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.892222 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.892311 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.892422 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":30}
I20260807 15:19:37.892987 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.882914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.31.230.252:45685)
0807 15:19:37.883202 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.883212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.883228 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:19:37.883264 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.883267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.883326 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.883417 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.883930 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.884711 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.885754 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.886115 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.886534 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.889080 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.889084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.889096 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.889122 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.890869 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.891370 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.891373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.891375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.891463 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.891745 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.891748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.891752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.892005 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.892094 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.892736 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.892839 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":125,"threads_started":1}
Aug 07 15:19:37 dist-test-slave-gkw7 krb5kdc[32383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.230.253@KRBTEST.COM
I20260807 15:19:37.893467 32504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:37.894452 32504 log.cc:824] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:37.896176 32747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: Bootstrap starting.
I20260807 15:19:37.897328 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626: No bootstrap required, opened a new log
I20260807 15:19:37.898924 32747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:37.899205 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.879594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.230.252:45685 (local address 127.0.0.1:47562)
0807 15:19:37.879786 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:19:37.879801 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:19:37.879870 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:19:37.881682 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:19:37.881686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:19:37.881914 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:19:37.882037 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.882047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.883790 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.883793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.884687 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:19:37.884698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.885423 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.885428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:19:37.885518 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.886044 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:19:37.886065 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:19:37.895151 (+ 9086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:19:37.897439 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.897443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.897446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.897749 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.898117 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:19:37.898120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:19:37.898122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:19:37.898223 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:19:37.898745 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:19:37.898749 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:19:37.898842 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 15:19:37.898933 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:19:37.899052 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 15:19:37.899134 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.887200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54624 (local address 127.31.230.253:33263)
0807 15:19:37.887785 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.887790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.887811 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:19:37.887841 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.887843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.887877 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.887986 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.888679 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.889304 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.890639 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.890914 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.891052 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.894136 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.894139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.894141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.894164 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.896436 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.897004 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.897011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.897015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.897129 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.897483 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.897490 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.897494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.897760 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.897978 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.898811 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.898944 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505}
I20260807 15:19:37.899642 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.879665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.31.230.252:45685)
0807 15:19:37.881356 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.881365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.881379 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:19:37.881420 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.881423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.881476 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.882183 (+ 707us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.882289 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.883651 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.884839 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.885286 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.885830 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.895290 (+ 9460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.895294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.895297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.895327 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.897306 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.897898 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.897901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.897902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.897995 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.898357 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.898360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.898362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.898612 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.899212 (+ 600us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.899304 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.899447 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1600,"thread_start_us":111,"threads_started":1}
I20260807 15:19:37.900612 32747 log.cc:824] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:37.901854 32504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.902081 32504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b697c40e41644f888019305a64e3626, State: Initialized, Role: FOLLOWER
I20260807 15:19:37.902392 32504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.902925 32760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.903055 32667 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:19:37.903061 32625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: Bootstrap starting.
I20260807 15:19:37.903172 32760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.903900 32504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:37.904364 32460 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1b697c40e41644f888019305a64e3626"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35018
I20260807 15:19:37.904618 32460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:19:37.904753 32460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:37.906071 32625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: Neither blocks nor log segments found. Creating new log.
I20260807 15:19:37.906270 32460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.907091 32625 log.cc:824] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:19:37.907914 32747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: No bootstrap required, opened a new log
I20260807 15:19:37.908505 32460 leader_election.cc:288] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [CANDIDATE]: Term 1 election: Requested vote from peers e0c6dc9ff96843489c954e21709ebb1f (127.31.230.253:33263), 205cd3b362744eba9eea3c6133498428 (127.31.230.252:45685)
W20260807 15:19:37.910270 307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:19:37.910338 307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:37.912611 32747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.912846 32747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205cd3b362744eba9eea3c6133498428, State: Initialized, Role: FOLLOWER
I20260807 15:19:37.913187 32747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.914777 32747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:37.910800 32401 master_runner.cc:186] Error getting master registration for 127.31.230.253:33263: OK, instance_id { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" instance_seqno: 1786115977619517 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:19:37.915413 32401 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:19:37.915546 32708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b697c40e41644f888019305a64e3626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "205cd3b362744eba9eea3c6133498428"
I20260807 15:19:37.915824 32708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:37.917335 32708 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b697c40e41644f888019305a64e3626 in term 1.
I20260807 15:19:37.917884 32423 leader_election.cc:302] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b697c40e41644f888019305a64e3626, 205cd3b362744eba9eea3c6133498428; no voters:
I20260807 15:19:37.918370 32760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:19:37.918617 32760 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [term 1 LEADER]: Becoming Leader. State: Replica: 1b697c40e41644f888019305a64e3626, State: Running, Role: LEADER
I20260807 15:19:37.918994 32760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.919407 32625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: No bootstrap required, opened a new log
I20260807 15:19:37.921020 32760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b697c40e41644f888019305a64e3626. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.921211 32760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:37.921663 340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:19:37.917876 318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
W20260807 15:19:37.921000 336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:19:37.922147 318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.922802 340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 15:19:37.922155 336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:37.925657 32625 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.925941 32625 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c6dc9ff96843489c954e21709ebb1f, State: Initialized, Role: FOLLOWER
I20260807 15:19:37.926294 32625 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } }
I20260807 15:19:37.927755 32625 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:19:37.927914 32708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [term 1 FOLLOWER]: Refusing update from remote peer 1b697c40e41644f888019305a64e3626: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:37.928591 32760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [LEADER]: Connected to new peer: Peer: permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:37.929649 32584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b697c40e41644f888019305a64e3626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e0c6dc9ff96843489c954e21709ebb1f"
I20260807 15:19:37.929895 32584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:19:37.931540 32584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b697c40e41644f888019305a64e3626 in term 1.
I20260807 15:19:37.931696 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.931828 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.933229 32584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [term 1 FOLLOWER]: Refusing update from remote peer 1b697c40e41644f888019305a64e3626: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:19:37.933893 32760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:19:37.936880 318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b697c40e41644f888019305a64e3626. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.937047 318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.937183 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.911135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54632 (local address 127.31.230.253:33263)
0807 15:19:37.911253 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.911258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.917863 (+ 6605us) server_negotiation.cc:407] Connection header received
0807 15:19:37.918090 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.918094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.918148 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.918203 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.925645 (+ 7442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.926388 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.927570 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.928065 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.928147 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.933103 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.933116 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.933119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.933146 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.935066 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.935583 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.935587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.935589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.935683 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.936014 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.936017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.936019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.936282 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.936424 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.936824 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.936964 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:19:37.926775 32635 master_runner.cc:186] Error getting master registration for 127.31.230.253:33263: OK, instance_id { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" instance_seqno: 1786115977619517 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:19:37.938340 32635 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:19:37.941953 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.913190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47576 (local address 127.31.230.252:45685)
0807 15:19:37.919853 (+ 6663us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.919858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.930742 (+ 10884us) server_negotiation.cc:407] Connection header received
0807 15:19:37.930925 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.930928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.930978 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.931031 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.932027 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.932769 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.935306 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.935719 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.935800 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.937518 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.937522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.937545 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.937571 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.939355 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.939910 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.939915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.939918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.940021 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.940383 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.940388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.940390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.940652 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.940820 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.941571 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.941716 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6557}
I20260807 15:19:37.944514 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b697c40e41644f888019305a64e3626. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.944672 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:37.912354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.31.230.254:35721)
0807 15:19:37.912470 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:37.912474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:37.929088 (+ 16614us) server_negotiation.cc:407] Connection header received
0807 15:19:37.929283 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:37.929288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:37.929343 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:37.929396 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:37.931535 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.932344 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.938565 (+ 6221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:37.938990 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:37.939086 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:37.940879 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:19:37.940883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:19:37.940886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:19:37.940922 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:19:37.942660 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.943182 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.943186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.943188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.943282 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:19:37.943641 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:19:37.943644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:19:37.943646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:19:37.943866 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:19:37.943994 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:37.944378 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 15:19:37.944505 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:19:37.944694 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.947770 32760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.947921 32760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:37.948062 342 mvcc.cc:205] Tried to move back new op lower bound from 7315931045586731008 to 7315931045560377344. Current Snapshot: MvccSnapshot[applied={T|T < 7315931045586731008}]
I20260807 15:19:37.948158 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.948282 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.948621 365 mvcc.cc:205] Tried to move back new op lower bound from 7315931045586731008 to 7315931045560377344. Current Snapshot: MvccSnapshot[applied={T|T < 7315931045586731008}]
I20260807 15:19:37.950492 32760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.950642 32760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626 [sys.catalog]: This master's current role is: LEADER
I20260807 15:19:37.951814 318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b697c40e41644f888019305a64e3626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b697c40e41644f888019305a64e3626" member_type: VOTER last_known_addr { host: "127.31.230.254" port: 35721 } } peers { permanent_uuid: "e0c6dc9ff96843489c954e21709ebb1f" member_type: VOTER last_known_addr { host: "127.31.230.253" port: 33263 } } peers { permanent_uuid: "205cd3b362744eba9eea3c6133498428" member_type: VOTER last_known_addr { host: "127.31.230.252" port: 45685 } } }
I20260807 15:19:37.951928 318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:19:37.952252 363 mvcc.cc:205] Tried to move back new op lower bound from 7315931045586731008 to 7315931045560377344. Current Snapshot: MvccSnapshot[applied={T|T < 7315931045586731008}]
I20260807 15:19:37.954250 340 catalog_manager.cc:1347] Generated new cluster ID: af9c9e2154394f03a830499f53d8fd2b
I20260807 15:19:37.954329 340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:19:37.959017 362 catalog_manager.cc:1259] Loaded cluster ID: af9c9e2154394f03a830499f53d8fd2b
I20260807 15:19:37.959084 362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: loading cluster ID for follower catalog manager: success
W20260807 15:19:37.959631 362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260807 15:19:37.959723 362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:19:37.968482 340 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:19:37.970088 340 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:19:37.984490 340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b697c40e41644f888019305a64e3626: Generated new TSK 0
I20260807 15:19:37.985638 340 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:19:38.026961 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.023322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.31.230.253:33263)
0807 15:19:38.023466 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.023471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.025682 (+ 2211us) server_negotiation.cc:407] Connection header received
0807 15:19:38.025914 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.025918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.025977 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.026037 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.026801 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54634: BlockingRecv error: recv got EOF from 127.0.0.1:54634 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 15:19:38.028898 32511 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.024284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.31.230.254:35721)
0807 15:19:38.024418 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.024423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.027757 (+ 3334us) server_negotiation.cc:407] Connection header received
0807 15:19:38.028037 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.028041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.028109 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.028167 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.028729 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35040: BlockingRecv error: recv got EOF from 127.0.0.1:35040 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 15:19:38.030901 32755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.024980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47580 (local address 127.31.230.252:45685)
0807 15:19:38.025115 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.025120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.029750 (+ 4630us) server_negotiation.cc:407] Connection header received
0807 15:19:38.029937 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.029941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.029997 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.030053 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.030775 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47580: BlockingRecv error: recv got EOF from 127.0.0.1:47580 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 15:19:38.036288 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.034506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.31.230.253:33263)
0807 15:19:38.034626 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.034631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.035181 (+ 550us) server_negotiation.cc:407] Connection header received
0807 15:19:38.035438 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.035442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.035533 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.035592 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.036170 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54642: BlockingRecv error: recv got EOF from 127.0.0.1:54642 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:19:38.042254 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.040465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.31.230.253:33263)
0807 15:19:38.040584 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.040589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.041173 (+ 584us) server_negotiation.cc:407] Connection header received
0807 15:19:38.041384 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.041388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.041446 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.041502 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.042136 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54656: BlockingRecv error: recv got EOF from 127.0.0.1:54656 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 15:19:38.048542 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.046867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.31.230.253:33263)
0807 15:19:38.046981 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.046985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.047541 (+ 556us) server_negotiation.cc:407] Connection header received
0807 15:19:38.047748 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.047752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.047808 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.047862 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.048426 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54672: BlockingRecv error: recv got EOF from 127.0.0.1:54672 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 15:19:38.057152 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.055281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.31.230.253:33263)
0807 15:19:38.055458 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.055463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.056049 (+ 586us) server_negotiation.cc:407] Connection header received
0807 15:19:38.056292 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.056296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.056354 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.056411 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.057004 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54680: BlockingRecv error: recv got EOF from 127.0.0.1:54680 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260807 15:19:38.070081 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.068131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.31.230.253:33263)
0807 15:19:38.068258 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.068262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.068894 (+ 632us) server_negotiation.cc:407] Connection header received
0807 15:19:38.069145 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.069151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.069213 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.069270 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.069955 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54696: BlockingRecv error: recv got EOF from 127.0.0.1:54696 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 15:19:38.091008 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.089084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.31.230.253:33263)
0807 15:19:38.089234 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.089239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.089912 (+ 673us) server_negotiation.cc:407] Connection header received
0807 15:19:38.090152 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.090155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.090216 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.090273 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.090883 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54704: BlockingRecv error: recv got EOF from 127.0.0.1:54704 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 15:19:38.127965 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.126129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.31.230.253:33263)
0807 15:19:38.126257 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.126262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.126895 (+ 633us) server_negotiation.cc:407] Connection header received
0807 15:19:38.127115 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.127119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.127173 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.127246 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.127840 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54712: BlockingRecv error: recv got EOF from 127.0.0.1:54712 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 15:19:38.196810 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.195452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.31.230.253:33263)
0807 15:19:38.195656 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.195661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.195829 (+ 168us) server_negotiation.cc:407] Connection header received
0807 15:19:38.196014 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.196017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.196078 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.196187 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.196681 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54714: BlockingRecv error: recv got EOF from 127.0.0.1:54714 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 15:19:38.328390 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.327203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.31.230.253:33263)
0807 15:19:38.327371 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.327374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.327510 (+ 136us) server_negotiation.cc:407] Connection header received
0807 15:19:38.327650 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.327653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.327702 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.327783 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.328289 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54730: BlockingRecv error: recv got EOF from 127.0.0.1:54730 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 15:19:38.587848 32633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:19:38.586759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54742 (local address 127.31.230.253:33263)
0807 15:19:38.586915 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:38.586918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:38.587024 (+ 106us) server_negotiation.cc:407] Connection header received
0807 15:19:38.587140 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:38.587142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:38.587197 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:38.587275 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:19:38.587704 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54742: BlockingRecv error: recv got EOF from 127.0.0.1:54742 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:19:38.926200 336 catalog_manager.cc:1259] Loaded cluster ID: af9c9e2154394f03a830499f53d8fd2b
I20260807 15:19:38.926295 336 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: loading cluster ID for follower catalog manager: success
I20260807 15:19:38.928020 336 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: acquiring CA information for follower catalog manager: success
I20260807 15:19:38.929311 336 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 205cd3b362744eba9eea3c6133498428: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:19:38.962297 362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: acquiring CA information for follower catalog manager: success
I20260807 15:19:38.963615 362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0c6dc9ff96843489c954e21709ebb1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:19:39.106928 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:39.102410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.31.230.253:33263)
0807 15:19:39.102686 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:39.102690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:39.102707 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:19:39.102824 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:39.102826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:39.102958 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:39.103061 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:39.103562 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.104154 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.105306 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.105607 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.105698 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:39.106003 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:39.106454 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:39.106547 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:39.106644 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:19:39.106715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":97,"threads_started":1}
I20260807 15:19:39.113736 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:39.110182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47586 (local address 127.31.230.252:45685)
0807 15:19:39.110425 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:39.110429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:39.110443 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:19:39.110546 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:39.110549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:39.110661 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:39.110764 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:39.110949 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.111468 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.112266 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.112481 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.112555 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:39.112826 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:39.113245 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:39.113338 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:39.113501 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 15:19:39.113577 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":104,"threads_started":1}
I20260807 15:19:39.121882 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:39.117346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.31.230.252:45685)
0807 15:19:39.117500 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:39.117505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:39.117865 (+ 360us) server_negotiation.cc:407] Connection header received
0807 15:19:39.117981 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:39.117984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:39.118028 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:39.118117 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:39.118442 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.119194 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.120537 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.120874 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.120972 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:39.121334 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:39.121424 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:39.121489 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:39.121666 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 15:19:39.121724 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:19:39.122030 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:39.116992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54760 (local address 127.31.230.253:33263)
0807 15:19:39.117261 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:39.117265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:39.117496 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:19:39.117764 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:39.117768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:39.117832 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:39.117905 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:39.118260 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.119075 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.120736 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.121070 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.121169 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:39.121506 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:39.121620 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:39.121722 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:39.121827 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:19:39.121881 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260807 15:19:39.123075 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:19:39.117015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.31.230.254:35721)
0807 15:19:39.117260 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 15:19:39.117266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:19:39.117304 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:19:39.117572 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:19:39.117579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:19:39.117764 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:19:39.118236 (+ 472us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:19:39.118930 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.119684 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.120697 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:19:39.121014 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:19:39.121126 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:19:39.121438 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:19:39.121858 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:19:39.122456 (+ 598us) server_negotiation.cc:1035] Waiting for connection context
0807 15:19:39.122868 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 15:19:39.122939 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
I20260807 15:19:39.132702 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 32401
I20260807 15:19:39.145670 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 32516
I20260807 15:19:39.156880 32667 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoCTfz/build/debug/bin/kudu with pid 32635
2026-08-07T15:19:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3325 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6824 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (298409 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST